Application Security

The State of Web Application and Data Security—Mid 2009

By Rich

One of the more difficult aspects of the analyst gig is sorting through all the information you get, and isolating out any inherent biases. The kinds of inquiries we get from clients can all too easily skew our perceptions of the industry, since people tend to come to us for specific reasons, and those reasons don't necessarily represent the mean of the industry. Aside from all the vendor updates (and customer references), our end user conversations usually involve helping someone with a specific problem -- ranging from vendor selection, to basic technology education, to strategy development/problem solving. People call us when they need help, not when things are running well, so it's all too easy to assume a particular technology is being used more widely than it really is, or a problem is bigger or smaller than it really is, because everyone calling us is asking about it. Countering this takes a lot of outreach to find out what people are really doing even when they aren't calling us.

Over the past few weeks I've had a series of opportunities to work with end users outside the context of normal inbound inquiries, and it's been fairly enlightening. These included direct client calls, executive roundtables such as one I participated in recently with IANS (with a mix from Fortune 50 to mid-size enterprises), and some outreach on our part. They reinforced some of what we've been thinking, while breaking other assumptions. I thought it would be good to compile these together into a "state of the industry" summary. Since I spend most of my time focused on web application and data security, I'll only cover those areas:

When it comes to web application and data security, if there isn't a compliance requirement, there isn't budget -- Nearly all of the security professionals we've spoken with recognize the importance of web application and data security, but they consistently tell us that unless there is a compliance requirement it's very difficult for them to get budget. That's not to say it's impossible, but non-compliance projects (however important) are way down the priority list in most organizations. In a room of a dozen high-level security managers of (mostly) large enterprises, they all reinforced that compliance drove nearly all of their new projects, and there was little support for non-compliance-related web application or data security initiatives. I doubt this surprises any of you.

"Compliance" may mean more than compliance -- Activities that are positioned as helping with compliance, even if they aren't a direct requirement, are more likely to gain funding. This is especially true for projects that could reduce compliance costs. They will have a longer approval cycle, often 9 months or so, compared to the 3-6 months for directly-required compliance activities. Initiatives directly tied to limiting potential data breach notifications are the most cited driver. Two technology examples are full disk encryption and portable device control.

PCI is the single biggest compliance driver for web application and data security -- I may not be thrilled with PCI, but it's driving more web application and data security improvements than anything else.

The term Data Loss Prevention has lost meaning -- I discussed this in a post last week. Even those who have gone through a DLP tool selection process often use the term to encompass more than the narrow definition we prefer.

It's easier to get resources to do some things manually than to buy a tool -- Although tools would be much more efficient and effective for some projects, in terms of costs and results, manual projects using existing resources are easier to get approval for. As one manager put it, "I already have the bodies, and I won't get any more money for new tools." The most common example cited was content discovery (we'll talk more about this a few points down).

Most people use DLP for network (primarily email) monitoring, not content discovery or endpoint protection -- Even though we tend to think discovery offers equal or greater value, most organizations with DLP use it for network monitoring.

Interest in content discovery, especially DLP-based, is high, but resources are hard to get for discovery projects -- Most security managers I talk with are very interested in content discovery, but they are less educated on the options and don't have the resources. They tell me that finding the data is the easy part -- getting resources to do anything about it is the limiting factor.

The Web Application Firewall (WAF) market and Security Source Code Tools markets are nearly equal in size, with more clients on WAFs, and more money spent on source code tools per client -- While it's hard to fully quantify, we think the source code tools cost more per implementation, but WAFs are in slightly wider use.

WAFs are a quicker hit for PCI compliance -- Most organizations deploying WAFs do so for PCI compliance, and they're seen as a quicker fix than secure source code projects.

Most WAF deployments are out of band, and false positives are a major problem for default deployments -- Customers are installing WAFs for compliance, but are generally unable to deploy them inline (initially) due to the tuning requirements.

Full drive encryption is mature, and well deployed in the early mainstream -- Full drive encryption, while not perfect, is deployable in even large enterprises. It's now considered a level-setting best practice in financial services, and usage is growing in healthcare and insurance. Other asset recovery options, such as remote data destruction and phone home applications, are now seen as little more than snake oil. As one CISO told us, "I don't care about the laptop, we just encrypt it and don't worry about it when it goes missing".

File and folder encryption is not in wide use -- Very few organizations are performing any wide scale file/folder encryption, outside of some targeted encryption of PII for compliance requirements.

Database encryption is hard, and not widely used -- Most organizations are dissatisfied with database encryption options, and do not deploy it widely. Within a large organization there is likely some DB encryption, with preference given to file/folder/media protection over column level encryption, but most organizations prefer to avoid it. Performance and key management are cited as the primary obstacles, even when using native tools. Current versions of database encryption (primarily native encryption) do perform better than older versions, but key management is still unsatisfactory. Large encryption projects, when initiated, take an average of 12-18 months.

Large enterprises prefer application-level encryption of credit card numbers, and tokenization -- When it comes to credit card numbers, security managers prefer to encrypt it at the application level, or consolidate numbers into a central source, using representative "tokens" throughout the rest of the application stack. These projects take a minimum of 12-18 months, similar to database encryption projects (the two are often tied together, with encryption used in the source database).

Email encryption and DRM tend to be workgroup-specific deployments -- Email encryption and DRM use is scattered throughout the industry, but is still generally limited to workgroup-level projects due to the complexity of management, or lack of demand/compliance from users.

Database Activity Monitoring usage continues to grow slowly, mostly for compliance, but not quickly enough to save lagging vendors -- Many DAM deployments are still tied to SOX auditing, and it's not as widely used for other data security initiatives. Performance is reasonable when you can use endpoint agents, which some DBAs still resist. Network monitoring is not seen as effective, but may still be used when local monitoring isn't an option. Network requirements, depending on the tool, may also inhibit deployments.

My main takeaway is that security managers know what they need to do to protect information assets, but they lack the time, resources, and management support for many initiatives. There is also broad dissatisfaction with security tools and vendors in general, in large part due to poor expectation setting during the sales process, and deliberately confusing marketing. It's not that the tools don't work, but that they're never quite as easy as promised.

It's an interesting dilemma, since there is clear and broad recognition that data security (and by extension, web application security) is likely our most pressing overall issue in terms of security, but due to a variety of factors (many of which we covered in our Business Justification for Data Security paper), the resources just aren't there to really tackle it head-on.

–Rich

New Application Security Certification Launched

By Rich

We've been talking a lot about application security since we started this blog, and one thing we've been tracking closely are training and certification programs. While we couldn't talk about it, we've been quietly involved with the Institute for Certified Application Security Specialists. We reviewed the program during development, and were overall pretty impressed. It has very similar requirements to the CSSLP, but is more cost effective for security practitioners... something we can all appreciate in this economy. Believe it or not, despite my not-infrequent diatribes against various certifications, I actually went through the process myself and am fully certified. What I really appreciate is how pragmatic the program is, and how it really reflects the operational realities of application security.

You can get more information at the Institute for Certified Application Security Specialists, and as a member of the affiliate program Securosis readers receive a 10% discount.

Oh- and don't forget to join the LinkedIn Group!

–Rich

Building a Web Application Security Program, Part 8: Putting It All Together

By Adrian Lane

'Whew! This is our final post in this series on Building a Web Application Security Program (Part 1, Part 2, Part 3, Part 4, Part 5, Part 6, Part 7), and it's time to put all the pieces together. Here are our guidelines for designing a program that meets the needs of your particular organization. Web application security is not a "one size fits all" problem. The risks, size, and complexity of the applications differ, the level of security awareness among team members varies, and most importantly the goals of each organization are different.

In order to offer practical advice, we needed to approach program development in terms of typical goals. We picked three use cases to represent common challenges organizations face with web app security, and will address those use cases with appropriate program models. We discuss a mid-sized firm tackling a compliance mandate for the first time, a large enterprise looking to improve security across customer-facing applications, and a mid-to-large organization dealing with security for internal applications. Each perspective has its own drivers and assumptions, and in each scenario different security measures are already in place, so the direction of each program will be different. Since we've been posting this over a series of weeks, before you dig in to this post we recommend you review Part 4: The Web Application Security Lifecycle which talks about all tools in all phases. First we describe the environment for each case, then overall strategy and specific recommendations.

Large Enterprise with Customer Facing Web Applications

For our first scenario, let's consider a large enterprise with multiple customer-facing web applications. These applications evolved to offer core business functions and are a principal contact point with customers, employees, and business partners. Primary business drivers for security are fraud reduction, regulatory compliance, and service reliability as tangible incentives. Secondary factors are breach preparedness, reputation preservation, and asset protection secondary -- all considerations for security spending. The question is not whether these applications need to be secured, but how. Most enterprises have a body of code with questionable security, and let's be totally honest here- these issues are flaws in your code. No single off-the-shelf product is going to magically make your application secure, so you invest not only in third-party security products, but also in improvements to your own development process which improve the product with each new release.

We assume our fictitious enterprise has an existing security program and the development team has some degree of maturity in their understanding of security issues, but how best to address problems is up for debate. The company will already have a 'security guy' in place, and while security is this guy's or gal's job, the development organization is not tasked with security assessments and problem identification. Your typical CISO comes from a network security background, lacks a secure code development background, and is not part of this effort. We find their security program includes vulnerability assessment tools, and they have conducted a review of the code for typical SQL injection and buffer overflow attacks. Overall, security is a combination of a couple third-party products and the security guy pointing out security flaws which are patched in upcoming release cycles.

Recommendations: The strategy is to include security within the basic development process, shifting the investment from external products to internal products and employee training. Tools are selected and purchased to address particular deficiencies in team skill or organizational processes. Some external products are retained to shield applications during patching efforts.

Training, Education, and Process Improvements: The area where we expect to see the most improvement is the skill and awareness of the web application development team. OWASP's top flaws and other sources point out issues that can be addressed by proper coding and testing ... provided the team knows what to look for. Training helps staff find errors and problems during code review, and iteratively reduces flaws through the development cycle. The development staff can focus on software security and not rely on one or two individuals for security analysis.

Secure SDLC: Knowing what to do is one thing, but actually doing it is something else. There must be an incentive or requirement for development to code security into the product, assurance to test for compliance, and product management to set the standards and requirements. Otherwise security issues get pushed to the side while features and functions are implemented. Security needs to be part of the product specification, and each phase of the development process should provide verification that the specification is being met through assurance testing. This means building security testing into the development process and QA test scenarios, as well as re-testing released code. Trained development staff can provide code analysis and develop test scripts for verification, but additional tools to automate and support these efforts are necessary, as we will discuss below.

Heritage Applications: Have a plan to address legacy code. One of the more daunting aspects for the enterprise is how to address existing code, which is likely to have security problems. There are several possible approaches for addressing this, but the basic steps are 1) identification of problems in the code, 2) prioritization on what to fix, and 3) planning how to fix individual issues. Common methods of addressing vulnerabilities include 1) rewriting segments of code, 2) method encapsulation, 3) temporary shielding by WAF ("secure & patch"), 4) moving SQL processing & validation into databases, 5) discontinuing use of insecure features, and 6) introduction of validation code within the execution path. We recommend static source code analysis or dynamic program analysis tools for the initial identification step. These tools are cost-effective and suitable for scanning large bodies of code to locate common risks and programming errors. They detect and prioritize issues, and reduce human error associated with tedious manual scanning by internal or external parties. Analysis tools also help educate staff about issues with certain languages and common programming patterns. The resulting arguments over what to do with 16k insecure occurrences of IFRAME are never fun, but acceptance of the problem is necessary before it can be effectively addressed.

External Validation: Periodic external review, through vulnerability assessment, penetration testing or source code review, is highly recommended . Skilled unbiased professionals with experience in threat analysis often catch items items which slip by internal scans, and can help educate development staff on different threat vectors. Plan on external penetration testing on a quarterly or biannual basis- their specific expertise and training goes far beyond the basics threats, and trained humans monitoring the output of sophisticated tools are very useful for detecting weaknesses that a hacker could exploit. We recommend the use of static testing tools for internal testing of code during the QA sweep, with internal penetration testing just prior to deployment so they can fully stress the application without fear of corrupting the production environment. Major releases should also undergo an external penetration test and review before deployment.

Blocking: This is one area that will really depend upon the specifics of your organization. In the enterprise use case, plan on using a Web Application Firewall. They provide basic protection and give staff a chance to remove security issues from the application. You may find that your code base is small and stable enough that you do not need WAF for protection, but for larger organizations this is not an option. Development and patching cycles are too long and cumbersome to counter threats in a reasonable timeframe. We recommend WAF + VA because in combination, they can relieve your organization from much of the threat research and policy development work for firewall rules. If your staff has the skill and time to develop WAF policies specific to your organization, you get customized policies at slightly greater expense in development costs. WAF isn't cheap, so we don't take this recommendation lightly, but it provides a great deal of flexibility in how and when threats are dealt with, today and as new threats evolve.

We recommend you take steps to improve security in every part of the development process. We are focused on improvements to the initial phases of development, as the impact of effort is greatest here, but we also recommend at the very least external assistance, and if budget allows, blocking. These later recommendations fill in other areas that need coverage, with penetration testing and web application firewalls. The risks to the enterprise are greater, the issues to overcome are more complex, and the corresponding security investment will therefore be larger. This workflow process should be formally documented for each stage of an application's lifecycle- from development through ongoing maintenance- with checkpoints for major milestones. Security shouldn't, and can't, be responsible for each stage, but should carry responsibility for managing the program and making sure the proper process is followed and maintained.

Mid-sized firm and PCI Compliance

If we are discussing web application security and compliance, odds are we are talking about the Payment Card Industry's Data Security Standard (PCI-DSS). No other compliance standard specifies steps to secure web applications like the PCI standard does. We can grouse about ambiguities and ways that it could be improved, but PCI is clearly the most widespread driver for web application security today, which is why our second use case is a mid-sized firm that needs to secure its web applications to satisfy PCI-DSS.

The profile for our company is a firm that generates a large portion of their revenue through Internet sales, and recent growth has made them a Tier 3 merchant. The commerce web site is relatively new (< 3 years) and the development team is small and not trained in security. Understanding the nuances of how criminals approach breaking code is not part of the team's skill set. PCI compliance is the mandate, and the team knows that they are both missing the basic requirements and susceptible to some kinds of attacks. The good news is that the body of code is small, and the web application accounts for a significant portion of the company's revenue, so management is supporting the effort.

In a nutshell, the PCI Data Security Standard is a security program specifically for companies that process credit card transactions for Internet commerce. In terms of compliance regulations, PCI-DSS requirements are clearer than most, making specific requirements for security tools and processes around credit card data. However, a company may also it has satisfy the spirit of the requirements in an alternate way, if it can demonstrate that the concern has been addressed. We will focus on the requirements outlined in sections 6.6 & 11.3, but will refer to sections 10 and compensating controls as well.

Recommendations: Our strategy focuses on education and process modifications to bring security into the development lifecycle. Additionally, we suggest assessment or penetration testing services to quickly identify areas of concern. Deploy WAF to address the PCI requirement immediately. Focus on the requirements to start, but plan for a more general program, and use compensating controls as your organization evolves. Use outside help and education to address immediate gaps, both in PCI compliance and more general application security.

Training, Education, and Process Improvements: Once again, we are hammering on education and training for the development team, including project management and quality assurance. While it takes time to come up to speed, awareness by developers helps keep security issues out of the code, and is cost-effective for securing the applications. Altering the process to accommodate fixing the code is essentially free, and code improvements become part of day to day efforts. With a small code base, education and training are easy ways to reap significant benefits as the company and code base grow.

External Help: Make friends with an auditor, or hire one as a consultant to help prepare and navigate the standard. While this is not a specific recommendation for any single requirement in PCI; auditors provide an expert perspective, help address some of the ambiguity in the standard, and assist in strategy and trade-off evaluations to avoid costly missteps.

Section 11.3.2: Section 11.3 mandates penetration testing of the network and the web application. In this case we recommend external penetration testing as an independent examination of the code. It is easy to recommend penetration testing, and not because it is required in the DSS specification, rather the independent & expert review of your application behavior will closely mimic the approach hackers will take. We also anticipate budget will require you make a choice between WAF and code reviews in section 6.6, so this will provide the necessary coverage. Should you use source code reviews, one could argue that acts as a compensating control for this section, but our recommendation is to stick with the external penetration testing. External testers provide much more than just a list of specific flaws, but also identify risky or questionable application behaviors in a near production environment.

Section 6.6: Our biggest debate internally was whether to recommend Web Application Firewall or expert code review to address section 6.6 of the PCI specification. The PCI Security Standards Council recommends that you do both, but it is widely recognized that this is prohibitively expensive. WAF provides a way to quickly meet the letter of 6.6's requirement, if not in spirit, provides basic monitoring, and is a flexible platform to block future attacks. The counter-arguments are significant and include cost, work required to customize policies for the application, and false positives & negatives. Alternatively, a code review by qualified security experts can identify weaknesses in application design and code usage and assist in education of the development team by pointing out specific flaws. Outside review is a very quick way to assess where you are and what you need. Down sides of review include cost, time to identify and fix errors, and that a constantly changing code base presents a moving target and thus requires repeated examinations.

Our recommendation here is deploy a WAF solution. Engaging a team of security professionals to review the code is an effective way to identify issues, but much of the value overlaps with the requirement of Section 11.3.2, periodic penetration testing of the application. The time to fix identified issues (even with a small-to-average body of code), with a development organization which is just coming to terms with security issues, is too long to meet PCI requirements in a timely fashion. Note that this recommendation is specific to this particular fictitious case- in other PCI audit scenarios, with a more experienced staff or a better handle on code quality, we might have made a different recommendation.

Monitoring: Database Activity Monitoring (DAM) is a good choice for Section 10 compliance- specifically by monitoring all access to credit card data. Web applications use a relational database back end to store credit card numbers, transactions, and card related data. DAM products that capture all network and console activity on the database platform provide a focused and cost-effective audit for all access to cardholder data. Consider this option for providing an audit trail for auditors and security personnel.

Internal Web Application Development

Our last use case is an internal web applications that serves employees and partners within a mid-to-large business. While this may not sound like a serious problem, given that companies have on average 1 internal application (web and traditional client/server) per 100 employees, even mid-sized companies have incredible exposure in this area. Using data from workflow, HR, accounting, business intelligence, sales, and other critical IT systems, these internal applications support employees and partners alike. And with access to pretty much all of the data within the company, security and integrity are major concerns. A common assumption is that these systems, behind the perimeter firewall, are not exposed to the same types of attacks as typical web applications, but this assumption has proven disastrous in many cases.

Investment here is motivated by fraud reduction, breach remediation, and avoidance of notification costs- and possibly by compliance. You may find this is difficult to sell to executive management if there is not a compliance mandate and hasn't been a previous breach, but if basic perimeter security is breached these applications need some degree of resiliency rather than blind confidence in network security and access control. TJ Maxx (http://www.tjx.com/) is an excellent illustration of the danger.

Strategy: Determine basic security of the internal application, fix serious issues, and leverage education, training, and process improvements to steadily improve the quality of code. We will assume that budgeting for security in this context is far smaller than for external-facing systems, so look to cooperate between groups and leverage tools and experience.

Vulnerability Assessment and Penetration Testing: Scanning web applications for significant security, patch and configuration issues is a recommended first step in determining if there are glaring issues. Assessment tools are a cost-effective way to establish baseline security and ensure adherence to minimum best practices. Internal penetration testing will help determine the overall potential risk and prioritization, but be extremely cautious of testing live applications.

Training, Education, and Process Improvements: These may be even more important in this scenario than in our other use cases, where the business justification provides incentive to invest in security, internal web applications may not get the same degree of attention. For these applications that have a captive audience, developers have greater controls over the types of environments that they support what can be required in terms of authentication. Use these freedoms to your advantage. Training should focus on common vulnerabilities within the application stack that is being used, and give critical errors the same attention that top priority bugs would receive. Verify that these issues are tested, either as part of the VA sweep, or a a component of regression testing.

Monitoring: Monitoring for suspicious activity and system misuse is a cost-effective way to detect issues and react to them. We find WAF solutions are often too expensive for deployment across hundreds of internal applications distributed across a company, and a more cost-effective approach to collecting and analyzing activity is highly recommended. Monitoring software that plugs into the web application is often very effective for delivering some intelligence at low cost, but the burden of analyzing the data then falls on development team members. Database Activity Monitoring can effectively focus on critical information at the back end and is more mature than Web Application Monitoring.

This segment of the series took much longer to write than we originally anticipated, as our research gave us conflicting answers to some questions, making our choices were far from easy. Our recommendations really depend upon the specifics of the situation and the organization. We approached this with use cases to demonstrate how the motivating factors; combined with the current state of web security; really guides the selection of tools, services, and process changes.

We found in every case that security as part of the overall development process is the most cost-effective and least disruptive to normal operations, and is our principal recommendation for each scenario. However, as transformation of a web application does not happen overnight, we rarely have the luxury of waiting for the development team to address all security issues is not realistic; in the meantime, external third-party services and products are invaluable for dealing with immediate security challenges.

–Adrian Lane

Building A Web Application Security Program: Part 7, Secure Operations

By Adrian Lane

We've been covering a heck of a lot of territory in our series on Building a Web Application Security Program (see Part 1, Part 2, Part 3, Part 4, Part 5, and Part 6). So far we've covered secure development and secure deployment, now it's time to move on to secure operations. This is the point where the application moves out of development and testing and into production.

Keep in mind that much of what we've talked about until now is still in full effect- just because you have a production system doesn't mean you throw away all your other tools and processes. Updates still need to go through secure development, systems and applications are still subject to vulnerability assessments and penetration testing (although you need to use a different process when testing live applications vs. staging), and configuration management and ongoing secure management are more important than ever before.

In the secure operations phase we add two new technology categories to support two additional processes- Web Application Firewalls (WAF) for shielding from certain types of attacks, and monitoring at the application and database levels to support auditing and security alerting.

Before we dig in, we also want to thank everyone who has been commenting on this series as we post it- the feedback is invaluable, and we're going to make sure everyone is credited once we put it into whitepaper format.

Web Application Firewalls (WAF)

The role of a web application firewall is to sit in front of or next to a web application, monitoring application activity, and alerting or blocking on policy violations. Thus it potentially serves two functions- as a detective control for monitoring web activity, and as a preventative control for blocking activity.

A web application firewall is a firewall specifically built to watch HTTP requests and block those that are malicious or don't comply with specific rules. The intention is to catch SQL injection, Cross Site Scripting (XSS), directory traversal, and various HTTP abuses, as well as authorization, request forgeries, and other attempts to alter web application behavior. The WAF rules and policies are effectively consistency checks, for both the HTTP protocol and application functionality. WAFs can alert or block activity based on general attack signatures (such as a known SQL injection attack for a particular database), or application-specific signatures for the web application being protected.

WAF products examine inbound and outbound HTTP requests, compare these with the firewall rules, and create alerts for conditions of concern. Finally, the WAF selects a disposition for the traffic: 1) let it pass, 2) let it pass but audit, 3) block the transaction, or 4) reset the connection.

WAFs typically network appliances. They are normally placed in-line as a filter for the application (proxy mode); or 'out-of-band', receiving traffic from a mirror or SPAN port. In the former scenario, all inbound and outbound requests are intercepted and inspected prior to the web server receiving the request or user receiving the response, reducing load on the web application. For SSL traffic, inline WAFs also need to proxy the SSL connection from the browser so it can decrypt and inspect traffic before it reaches the web server, or after it leaves the web server for responses. In out-of-band mode, there are additional techniques to monitor the encrypted connections by placing a copy of the server certificate on the WAF, or positioning it behind an SSL concentrator. Some vendors also provide WAF capabilities via plug-ins for specific platforms, rather than through external devices.

The effectiveness of any WAF is limited by the quality of the policies it is configured to enforce. Policies are important not merely to ability to recognize and stop known/specific attacks, but also for flexibly dealing with ambiguous and unknown threat types, while keeping false positives manageable and without preventing normal transaction processing. The complexity of the web application, combined with the need for continuous policy updates, and the wide variety of deployment options to accommodate, pose a complex set of challenges for any WAF vendor. Simply dropping a WAF in front of your application and turning on all the default rules in blocking mode is a recipe for disaster. There is no way for black box to effectively understand all the intricacies of a custom application, and customization and tuning are essential for keeping false positives and negatives under control.

When deployed in monitoring mode, the WAF is used in a manner similar to an intrusion detection system (IDS). It's set to monitor activity and generate alerts based on policy violations. This is how you'll typically want to initially deploy the WAF, even if you plan on blocking activity later. It gives you an opportunity to tune the system and better understand application activity before you start trying to block connections. An advantage of monitoring mode is that you can watch for a wider range of potential attacks without worrying that false positives will result in inappropriate blocking. The disadvantages are 1) your incident handlers will spend more time dealing with these incidents and false positives, and 2) bad activity won't be blocked immediately.

In blocking/enforcement mode, the WAF will break connections by dropping them (proxy mode) or sending TCP reset packets (out of band mode) to reset the connection. The WAF can then ban the originating IP, permanently or temporarily, to stop additional attacks from that origin. Blocking mode is most effective when deployed as part of a "shield then patch" strategy to block known vulnerabilities in your application.

When a vulnerability is discovered in your application, you build a specific signature to block attacks on it and deploy that to the WAF (the "shield"). This protects your application as you go back and fix the vulnerable code, or wait for an update from your software provider (the "patch"). The shield then patch strategy greatly reduces potential false positives that interfere with application use and improves performance, but is only possible when you have adequate processes to detect and evaluate these vulnerabilities.

You can combine both approaches by deploying a larger signature set in monitoring mode, but only enabling a few specific policies in blocking mode.

Given these challenges, satisfaction with WAF products varies widely among security professionals who use them. While WAFs are effective against known threats, they are less capable of discovering new issues or handling questionable use cases. Some WAF products are addressing these issues by linking web application firewalls more tightly to the vulnerability assessment process and tools, as we'll discuss in a moment.

Monitoring

Monitoring is primarily used for discovery, both of how an application is used by real users, and also for how it can be misused. The fundamental value of monitoring is to learn what you do not already know- this is important not only for setting up a WAF, but also for tune an application security strategy. Although WAFs provide some level of application activity monitoring, there are three additional ways to monitor web applications, each with a different perspective on application activity:

• Network Monitoring: Monitoring network activity between the user/Internet and the web server. This category includes web application firewalls, intrusion detection systems, packet sniffers, and other external tools. While generic network security and sniffing tools can capture all network traffic, they have much less capability to place it in context and translate network activity into application transactions. Simply viewing HTTP traffic is often insufficient for understanding what users are attempting in an application- this is where interpretation is required. If a solution includes web application specific analysis and the ability to (potentially) audit all web activity, we call it Web Application Monitoring (WAM). While network monitoring is easy to implement and doesn't require application changes, it can only monitor what's going into and coming out of the application. This may be useful for detecting traditional attacks against the application stack, but much less useful than seeing traffic fully correlated to higher-level transactions.
• Application Auditing/Logging: Collection and analysis of application logs and internal activity. Both custom and off-the-shelf applications often include internal auditing capabilities, but there is tremendous variation in what's captured and how it's formatted and stored. While you gain insight into what's actually occurring in the application, not all applications log equally (or at all)- you are limited to whatever the programmers decided to track. For major enterprise applications, such as SAP, we've seen third party tools that either add additional monitoring or can interpret native audits. Log management and SIEM tools can also be used to collect and interpret (to a more limited degree) application activity when audit logs are generated.
• Database Activity Monitoring: DAM tools use a variety of methods to (potentially) record all database transactions and generate alerts on specific policy violations. By monitoring activity between the application and the database, DAM can provide a more precise examination of data elements, and awareness of multi-step transactions which directly correspond to business functions. Some DAM tools have specific plug ins for major application types (e.g., SAP & PeopleSoft) to translate database transactions into application activity. Since the vast majority of web applications run off databases, this is an effective point to track activity and look for policy violations. A full discussion of DAM is beyond the scope of this post, but more information is available in our DAM whitepaper.

WAF can be used for monitoring, but these alternative tools offer a wider range of activity collection, which helps to detect probing which may not be obvious from HTTP requests alone. The focus of web application monitoring is to examine behavior across data sources and provide analysis, recording activity trails and alerting on suspicious events, whereas WAFs are more focused on detection and blocking of known threats. There are significant differences between WAMs and WAFs in areas such as activity storage, aggregation of data from multiple sources, and examination of the collected data, so choosing the best tool depends specifics of the requirement. We must point out that web application monitoring products are not fully mature, and the handful of available products are early in the product evolution cycle.

WAF + VA

Several vendors have begun providing a hybrid model that combines web application vulnerability assessment with a web application firewall. As mentioned earlier, one of the difficulties with a shield then patch strategy is detecting vulnerabilities and building the WAF signatures to defend them. Coupling assessment tools or services with WAFs by feeding the assessment results to the firewall, and having the firewall adjust its policy set accordingly, can makes the firewall more effective. The intention is to fill the gap between exploits discovery/announcement and deployment of tested patches in the application, by instructing the WAF to block access to the particular weakness. In this model the assessment determines that there is a vulnerability and feeds the information to the WAF. The assessment policy contains WAF instructions on what to look for and how to respond. The WAF then dynamically incorporates the policy and protects the application.

This is the last post in this series where we discuss your options at different stages of the application lifecycle. Our next post will discuss which options you should consider, and how to balance your resource expenditures into an overall program.

–Adrian Lane

How The Cloud Destroys Everything I Love (About Web App Security)

By Rich

On Tuesday, Chris Hoff joined me to guest host the Network Security Podcast and we got into a deep discussion on cloud security. And as you know, for the past couple of weeks we've been building our series on web application security. This, of course, led to all sorts of impure thoughts about where things are headed. I wouldn't say I'm ready to run around in tattered clothes screaming about the end of the Earth, but the company isn't called Securosis just because it has a nice ring to it.

If you think about it a certain way, cloud computing just destroys everything we talk about for web application security. And not just in one of those, "oh crap, here's one of those analysts spewing BS about something being dead" ways. Before jumping into the details, in this case I'm talking very specifically of cloud based computing infrastructure- e.g., Amazon EC2/S3. This is where we program our web applications to run on top of a cloud infrastructure, not dedicated resources in a colo or a "traditional" virtual server. I also sprinkle in cloud services- e.g., APIs we can hook into using any application, even if the app is located on our own server (e.g., Google APIs).

Stealing from our yet incomplete series on web app sec and our discussions of ADMP, here's what I mean:

• Secure development (somewhat) breaks: we're now developing on a platform we can't fully control- in a development environment we may not be able to isolate/lock down. While we should be able to do a good job with our own code, there is a high probability that the infrastructure under us can change unexpectedly. We can mitigate this risk more than some of the other ones I'll mention- first, through SLAs with our cloud infrastructure provider, second by adjusting our development process to account for the cloud. For example, make sure you develop on the cloud (and secure as best you can) rather than completely developing in a local virtual environment that you then shift to the cloud. This clearly comes with a different set of security risks (putting development code on the Internet) that also need to be, and can be, managed. Data de-identification becomes especially important.
• Static and dynamic analysis tools (mostly) break: We can still analyze our own source code, but once we interact with cloud based services beyond just using them as a host for a virtual machine, we lose some ability to analyze the code (anything we don't program ourselves). Thus we lose visibility into the inner workings of any third party/SaaS APIs (authentication, presentation, and so on), and they are likely to randomly change under our feet as the providing vendor continually develops them. We can still perform external dynamic testing, but depending on the nature of the cloud infrastructure we're using we can't necessarily monitor the application during runtime and instrument it the same way we can in our test environments. Sure, we can mitigate all of this to some degree, especially if the cloud infrastructure service providers give us the right hooks, but I don't hold out much hope this is at the top of their priorities. (Note for testing tools vendors- big opportunity here).
• Vulnerability assessment and penetration testing... mostly don't break: So maybe the cloud doesn't destroy everything I love. This is one reason I like VA and pen testing- they never go out of style. We still lose some ability to test/attack service APIs.
• Web application firewalls really break: We can't really put a box we control in front of the entire cloud, can we? Unless the WAF is built into the cloud, good luck getting it to work. Cloud vendors will have to offer this as a service, or we'll need to route traffic through our WAF before it hits the back end of the cloud, negating some of the reasons we switch to the cloud in the first place. We can mitigate some of this through either the traffic routing option, virtual WAFs built into our cloud deployment (we need new products for it), or cloud providers building WAF functionality into their infrastructure for us.
• Application and Database Activity Monitoring break: We can no longer use external monitoring devices or services, and have to integrate any monitoring into our cloud-based application. As with pretty much all of this list it's not an impossible problem, just one people will ignore. For example, I highly doubt most of the database activity monitoring techniques will work in the cloud- network monitoring, memory monitoring, or kernel extensions. Native audit might, but not all database management systems provide effective audit logs, and you still need a way to collect them as your app and db shoot around the cloud for resource optimization.

I could write more about each of these areas, but you get the point. When we run web applications on cloud based infrastructure, using cloud based software services, we break much of the nascent web application security models we're just starting to get our fingers around. The world isn't over*, but it sure just moved out from under our feet.

*This doesn't destroy the world, but it's quite possible that the Keanu Reeves version of The Day the Earth Stood Still will.

–Rich

The Biggest Difference Between Web Applications And Traditional Applications.

By Rich

Adrian and I have been hard at work on our web application security overview series, and in a discussion we realized we left something off part 3 of the series when we dig into the differences between web applications and traditional applications.

In most applications we program the user display/interface. With web applications, we rely on an external viewer (the browser) we can't completely control, that can be interacting with other applications at the same time.

Which is stupid, because it's the biggest, most obvious difference of them all.

–Rich

Building A Web Application Security Program: Part 2, The Business Justification

By Adrian Lane

'In our last post in this series we introduced some of the key reasons web application security is typically underfunded in most organizations. The reality is that it's often difficult to convince management why they need additional protections for an application that seems to be up and running just fine. Or to change a development process the developers themselves are happy with. While building a full business justification model for web application security is beyond the scope of this post (and worthy of its own series), we can't talk about building a program without providing at least some basic tools to determine how much you should invest, and how to convince management to support you. The following list isn't a comprehensive business justification model, but provides typical drivers we commonly see used to justify web application security investments:

Compliance - Like it or not, sometimes security controls are mandated by government regulation, industry standards/requirements, or contractual agreements. We like to break compliance into three separate justifications- mandated controls (PCI web application security requirements), non-mandated controls that avoid other compliance violations (data protection to avoid a breach disclosure), and investments to reduce the costs of compliance (lower audit costs or TCO). The average organization uses all three factors to determine web application security investments.

Fraud Reduction - Depending on your ability to accurately measure fraud, it can be a powerful driver of, and justification for, security investments. In some cases you can directly measure fraud rates and show how they can be reduced with specific security investments. Keep in mind that you may not have the right infrastructure to detect and measure this fraud in the first place, which could provide sufficient justification by itself. Penetration tests are also useful is justifying investments to reduce fraud- a test may show previously unknown avenues for exploitation that could be under active attack, or open the door to future attack. You can use this to estimate potential fraud and map that to security controls to reduce losses to acceptable levels.

Cost Savings - As we mentioned in the compliance section, some web application security controls can reduce your cost of compliance (especially audit costs), but there are additional opportunities for savings. Web application security tools and processes during the development and maintenance stages of the application can reduce costs of manual processes or controls and/or costs associated with software defects/flaws, and may cause general efficiency improvements. We can also include cost savings from incident reduction- including incident response and recovery costs.

Availability - When dealing with web applications, we look at both total availability (direct uptime), and service availability (loss of part of the application due to attack or to repair a defect). For example, while it's somewhat rare to see a complete site outage due to a web application security issue (although it definitely happens), it's not all that rare to see an outage of a payment system or other functionality. We also see cases where, due to active attack, a site needs to shut down some of its own services to protect users, even if the attack didn't break the services directly.

User Protection - While this isn't quantifiable with a dollar amount, a major justification for investment in web security is to protect users from being compromised by their trust in you (yes, this has reputation implications, but not ones we can precisely measure). Attackers frequently compromise trusted sites not to steal from that site, but to use it to attack the site's users. Even if you aren't concerned with fraud resulting in direct losses to your organization, it's a problem if your web application is used to defraud your users.

Reputation Protection - While many models attempt to quantify a company's reputation and potential losses due to reputation damage, the reality is all those models are bunk- there is no accurate way to measure the potential losses associated with a successful attack. Despite surveys indicating users will switch to competitors if you lose their information, or that you'll lose future business, real world stats show that user behavior rarely aligns with survey responses. For example, TJX was the largest retail breach notification in history, yet sales went up after the incident. But just because we can't quantify reputation damage doesn't mean it isn't an important factor in justifying web application security. Just ask yourself (or management) how important that application is to the public image of your organization, and how willing you or they are to accept the risk of losses ranging from defacement, to lost customer information, to downtime.

Breach Notification Costs - Aside from fraud, we also have direct losses associated with breach notifications (if sensitive information is involved). Ignore all the fluffy reputation/lost business/market value estimates and focus on the hard dollar costs of making a list, sending a notification, and manning the call center for customer inquiries. You might also factor in the cost of credit monitoring, if you'd offer that to your customers.

You'll know which combination of these will work best for you based on your own organizational needs and management priorities, but the key takeaway should be that you likely need to mix quantitative and qualitative assessments to prioritize your investments. If you're dealing with private information (financial/retail/healthcare), compliance drivers and breach notification mixed with cost savings are your best option. For general web services, user protection & reputation, fraud reduction, and availability are likely at the top of your list. And let's not forget many of these justifications are just as relevant for internal applications.

Whatever your application, there is no shortage of business (not technical) reasons to invest in web application security.

–Adrian Lane

Common Applications Are Now The Weakest Link

By Rich

Edited: I stupidly credited Nate Lawson for Mark Dowd’s work with Sotirov. Dumb mistake, and I apologize.

Since my travel is slowing down a bit, I’m finally able catch up a little on my reading. Two articles this week reminded me of something I’ve been meaning to talk about. First, Chris Wysopal talks about how we’ve reached an application security tipping point. How the OS vendors are doing such a (relatively) good job at hardening the operating system that it’s become easier and more lucrative for attackers to go after common applications. Since nearly everyone online has a reasonably common set of Internet-enabled desktop apps running, it’s nearly as effective as targeting the OS. Heck, in some cases these apps are cross platform, and in a few cases we even see cross platform exploits.

To top it off, many of these applications do not activate or use anti-exploitation features like ASLR or DEP, even when it’s little more than a checkbox during the development process. Thus, as we saw during Alex Sotirov and Mark Dowd’s demo at Black Hat this year, you can use these applications to totally circumvent host operating system security, even through the web browser.

As Chris states:

Whoa. Millions of dollars spent on securing the most prevalent piece of software and it could be meaningless? Yes, it's true. Since attackers typically only need one vulnerability, if it isn"t in the network, and it isn"t in the host configuration, and it isn"t in the OS, they will happily exploit a vulnerability in an application.

Mike Andrews also nails it:

They don't just go away, they go to the next level of lowest hanging fruit. It might be other vendors (Apple, Adobe, Google for example) which may not have the focus that Microsoft has been forced to have, or even worse, smaller players like custom websites or things like Wordpress, Movable Type, phpbb, vbulletin, etc — software that has a huge install base, but perhaps not the resources to deal with a full-frontal attack. ...snip... Think of it security"s own Hydra — cut of one head (vulns in a major vendor), two grow back (vulns in smaller vendors), and that"s a worrying proposition.

As I often say on this blog we have a term for this... it's called job security.

–Rich

WAF vs. Secure Code vs. Dead Fish

By Rich

I've been slowly catching up on my reading after months of near-nonstop travel, and this post over at Imperviews caught my eye. Ignoring the product promotion angle, it raises one of my major pet peeves these days. I'm really tired of the Web Application Firewall vs. secure coding debate, never mind using PCI 6.6 to justify one over the other for security effectiveness. It's like two drunk cajuns arguing over the relative value of shrimp or pork in gumbo- you need both, and if either is spoiled the entire thing tastes like sh&t. You also can't dress up the family dog and fish in a pinch, use them as substitutes, and expect your kids to appreciate either the results or use of resources (resulting gumbo or the loss of Rover).

Here's the real deal-

Secure coding is awesome and you need to adopt a formal process if you produce any meaningful volume of code. But it takes a ton of resources to get to the old code (which you should still try to do), and can't account for new vulnerability classes. Also, people screw up... even when there are multiple layers to detect or prevent them from screwing up.

On the other hand, WAFs need to get a hell of a lot better. We're seeing some positive advancements, as I've written about before, but they still can't stop all vulnerabilities, can't stop logic flaws and certain other categories of attack, can't deal with the browser end, and I hear a lot of complaints about tuning (while I think liking WAFs with Vulnerability Assessment is a great start on this problem, we're just at the start of that race).

I absolutely hate to tell you to buy more than you need, but if you have a major web presence you likely need both these days, in the right combination (plus a few other things).

If you don't have the resources for both, I suggest two options. First, if you are really on the low end of resources, use hosted applications and standard platforms as much as possible to limit your custom coding. Then, make sure you have kick ass backups. Finally, absolutely minimize the kinds of information and transaction you expose to the risk of web attacks- drop those ad banners, minimize collecting private information, and validate transactions on the back end as much as possible.

If you do have some more resources available, I suggest starting with a vulnerability assessment (not a cheap ass bare-bones PCI scan, but something deeper), and using that to figure out where to go next.

Yes- we are eating our own dog food on this one. The blog is hosted using a standard platform. We know it's vulnerable, so we've minimized the attack surface as best we can and make sure we have backups of all the content. I've been pleasantly surprised we haven't been nailed yet, but I expect it to happen eventually. None of our sensitive operations are on that server, and we've pulled email and our other important stuff in house.

Early next year we're going to be launching some new things, and we will again go with remote hosting (on a more powerful platform). This time, we are switching to a more secure platform than Wordpress (Expression Engine) and will pay for a full vulnerability assessment and penetration test (at least annually, or when any major new components come online). We may perform some financial transactions, and we'll use an external provider for that. A WAF is out of budget for us, so we'll focus on minimizing our exposure and manually fixing problems discovered by ongoing assessments. We also plan on using as little custom code as possible.

But seriously- I'm tired of this debate. Both options have value, they aren't exclusionary, and which you need depends on what you are doing and how many resources you have.

Eventually we'll get a better lock on this problem, but that's a few years out.

–Rich

The Future Of Application And Database Security: Part 2, Browser To WAF/Gateway

By Rich

Since Friday is usually "trash" day (when you dump articles you don't expect anyone to read) I don't usually post anything major. But thanks to some unexpected work that hit yesterday, I wasn't able to get part 2 of this series out when I wanted to. If you can tear yourself away from those LOLCatz long enough, we're going to talk about web browsers, WAFs, and web application gateways. These are the first two components of Application and Database Monitoring and Protection (ADMP), which I define as:

Products that monitor all activity in a business application and database, identify and audit users and content, and, based on central policies, protect data based on content, context, and/or activity.

Browser Troubles

As we discussed in part 1, one of the biggest problems in web application security is that the very model of the web browsers and the World Wide Web is not conducive to current security needs. Browsers are the ultimate mashup tool- designed to take different bits from different places and seamlessly render them into a coherent whole. The first time I started serious web application programming (around 1995/96) this blew my mind. I was able to embed disparate systems in ways never before possible. And not only can we embed content within a browser, we can embed browsers within other content/applications. The main reason, as a developer, I converted from Netscape to IE was that Microsoft allowed IE to be embedded in other programs, which allowed us to drop it into our thick VR application. Netscape was stand alone only; seriously limiting its deployment potential.

This also makes life a royal pain on the security front where we often need some level of isolation. Sure, we have the same-origin policy, but browsers and web programming have bloated well beyond what little security that provides. Same-origin isn't worthless, and is still an important tool, but there are just too many ways around it. Especially now that we all use tabbed browsers with a dozen windows open all the time. Browsers are also stateless by nature, no matter what AJAX trickery we use. XSS and CSRF, never mind some more sophisticated attacks, take full advantage of the weak browser/server trust models that result from these fundamental design issues.

In short, we can't trust the browser, the browser can't trust the server, and individual windows/tabs/sessions in the browser can't trust each other. Fun stuff!

WAF Troubles

I've talked about WAFs before, and their very model is also fundamentally flawed. At least how we use WAFs today. The goal of a WAF is, like a firewall, to drop known bad traffic or only allow known good traffic. We're trying to shield our web applications from known vulnerabilities, just like we use a regular firewall to block ports, protocols, sources, and destinations. Actually, a WAF is closer to IPS than it is to a stateful packet inspection firewall.

But web apps are complex beasts; every single one a custom application, with custom vulnerabilities. There's no way a WAF can know all the ins and outs of the application behind it, even after it's well tuned. WAFs also only protect against certain categories of attacks- mostly some XSS and SQL injection. They don't handle logic flaws, CSRF, or even all XSS. I was talking with a reference yesterday of one of the major WAFs, and he had no trouble slicing through it during their eval phase using some standard techniques.

To combat this, we're seeing some new approaches. F5 and WhiteHat have partnered to feed the WAF specific vulnerability information from the application vulnerability assessment. Imperva just announced a similar approach, with a bunch of different partners.

These advances are great to see, but I think WAFs will also need to evolve in some different ways. I just don't think the model of managing all this from the outside will work effectively enough.

Enter ADMP

The idea of ADMP is that we build a stack of interconnected security controls from the browser to the database. At all levels we both monitor activity and include enforcement controls. The goal is to start with browser session virtualization connected to a web application gateway/WAF. Then traffic hits the web server and web application server, both with internal instrumentation and anti-exploitation. Finally, transactions drop to the database, where they are again monitored and protected.

All of the components for this model exist today, so it's not science fiction. We have browser session virtualization, WAFs, SSL-VPNs (that will make sense in a minute), application security services and application activity monitoring, and database activity monitoring. In addition to the pure defensive elements, we'll also tie in to the applications at the design and code level through security services for adaptive authentication, transaction authentication, and other shared services (happy Dre? :) ). The key is that this will all be managed through a central console via consistent policies.

In my mind, this is the only thing that makes sense. We need to understand the applications and the databases that back them. We have to do something at the browser level since even proper parameterization and server side validation can't meet all our needs. We have to start looking at transactions, business context, and content, rather than just packets and individual requests.

Point solutions at any particular layer have limited effectiveness. But if we stop looking at our web applications as pieces, and rather design security that addresses them as a whole, we'll be in much better shape. Not that anything is perfect, but we're looking at risk reduction, not risk elimination. A web application isn't just a web server, just some J2EE code, or just a DB- it's a collection of many elements working together to perform business transactions, and that's how we need to look at them for effective security.

The Browser and Web Application Gateway

A little while back I wrote about the concept of browser session virtualization. To plagiarize myself and save a little writing time so I can get my behind to happy hour:

What we ideally need is a way to completely isolate our content in the browser. One way to do this is session virtualization, pioneered by GreenBorder, who was later acquired by Google (the GreenBorder site is just in support mode now). When a user connects to our site, we push down some code to create a virtual environment in the browser that we strictly control. We wall off that session, which could just be an isolated iFrame in a page, so that it only accesses content we send it. Basically, we break the normal browser model and hijack what we need. This would, for example, help stop CSRF since other browser elements won't be able to trigger a connection to our application. Done right, it limits man in the middle attacks, even if the user authorizes a bad digital certificate. To work properly, this needs to be tied to a gateway that controls the session. While we could do it from the web/app server itself, I suspect we'll see this as a web application firewall feature, just as we see similar features from SSL-VPNs. I think isolated WAFs have a very limited lifespan, but this is exactly the kind of feature that will extend their value. Better yet, we can tie this in to our Application and Database Monitoring and Protection to build a browser-to-database protected path. We can completely track a transaction or piece of content from the database server to the browser and back. We could even use this to isolate out potentially "bad" content in an in-browser sandbox. For example, it could be a way to enable all those social networking widgets in a more controlled way but locking in potentially bad content instead of known good. Will this protect us from keystroke sniffers or a completely compromised host? Nope, but it will definitely help with a large number of our current browser security issues. If we combine it with full ADMP and additional methods like transaction authentication, I think we can regain a bit of control of the web application security mess.

Thus we see one migration path for a WAF. A user goes to connect to the application and hits the WAF, which is now more of a Web Application Gateway. The gateway, like an SSL-VPN, sends the session virtualization code down to the browser. We do this outside of the web application for performance reasons. The secure virtual session is established and the gateway then allows communications with the application behind it.

For things like retail and financial sites that include only limited third party content if any, we can monitor activity from the browser through to the application and work within the isolated session. It both improves our ability to control what's being sent to the browser, and gives us a higher degree of confidence that what's coming from the browser is safe. We still validate everything, but since we're tied to the application itself we can validate in the browser and at the gateway before we even hit the app (and further validate there). Since in a controlled environment we know what transactions should be allowed (or not), we have more ability to detect and block "bad" transactions like SQL injection from the user.

In less controlled environments- thing MySpace or Gmail and everything in between- the gateway also becomes a filter for third party content. Like Checkpoint's new ForceField. The gateway filters out, to the best of its ability, harmful third party content coming from third party sites. Basically, it becomes an SSL-VPN for secure browsing.

This is obviously not viable for all sites due to bandwidth considerations, and in those circumstances we'll drop this part and stick to the rest of the ADMP stack, or only virtualize our pieces of content knowing the user is at risk for the third party stuff we're still linking them to.

Future of the WAF, Option 2

I've just described a scenario where the WAF extends into a Secure Web Application Gateway that adds virtualization, encryption, and content filtering. That doesn't mean WAFs won't also still exist in non-virtualized situations, since that will still be a massive volume of sites out there.

For these sites the WAF continues to progress with deeper application integration and application understanding, and works with the elements I'll describe later that will be embedded into the applications and databases. Rather than hanging around outside the application with barely any idea what's going on behind it, the WAF will take its cues from the app, help manage sessions, and monitor activity outside the app to block the few things we know we can pick up at that layer.

Why use the WAF at all? To give us a choke point and offload some of the monitoring and processing that could hurt application performance. Let's be honest- maybe WAFs will eventually go away, but performance problems alone will probably keep next-gen WAFs viable for a while. There are also plenty of things we can now block before they ever hit the application controls, which, by nature of being integrated at the app level, will be more complex and delicate.

But again, by tightly integrating with our other layers, instead of naievely assuming that an external black box will magically solve our problems, we get a much higher level of functionality. Feeding in vulnerability data as we're just starting to do is a good beginning, but once we plug in deeper to the application and database servers we'll get entirely new levels of functionality.

Part 2 Conclusions

What I've described today is how we can build a (more) trusted path from the browser to the face of the application. WAFs will add gateway capabilities, both protecting the applications behind them and the browsers in front of them. Since this won't be the right approach in all circumstances, WAFs will also evolve with tighter integration to the application and other ADMP stack components.

Again, this might sound like little more than the usual analyst fiction, but all the components are here today. Also, I don't expect my predictions to be totally accurate. I'm roughly guessing I'm at 85% or so.

Next week I'll start digging into the application and database. We'll talk about application instrumentation, anti-exploitation, DAM, trusted transaction paths, and shared security services.

–Rich

The Future Of Application And Database Security: Part 1, Setting The Stage

By Rich

I've been spending the past few weeks wandering around the country for various shows, speaking to some of the best and brightest in the world of application and database security. Heck, I even hired one of them. During some of my presentations I laid out my vision for where I believe application (especially web application) and database security are headed. I've hinted at it here on the blog, discussing the concepts of ADMP, the information-centric security lifecycle, and DAM, but it's long past time I detailed the big picture.

I'm not going to mess around and write these posts so they are accessible to the non-geeks out there. If you don't know what secure SDLC, DAM, SSL-VPN, WAF, and connection pooling mean, this isn't the series for you. That's not an insult, it's just that this would drag out to 20+ pages if I didn't assume a technical audience.

Will all of this play out exactly as I describe? No way in hell. If everything I predict is 100% correct I'm just predicting common knowledge. I'm shooting for a base level of 80% accuracy, with hopes I'm closer to 90%. But rather than issuing some proclamation from the mount, I'll detail why I think things are going where they are. You can make your own decisions as to my assumptions and the accuracy of the predictions that stem from them.

Also, apologies to Dre's friends and family. I know this will make his head explode, but that's a cost I'm willing to pay. Special thanks to Chris Hoff and the work we've been doing on disruptive innovation, since that model drives most of what I'm about to describe. Finally, this is just my personal opinion as to where things will go. Adrian is also doing some research on the concept of ADMP, and may not agree with everything I say. Yes, we're both Securosis, but when you're predicting uncertain futures no one can speak with absolute authority. (And, as Hoff says, no one can tell you you're wrong today).

Forces and Assumptions

Based on the work I've been doing with Hoff, I've started to model future predictions by analyzing current trends and disruptive innovations. Those innovations that force change, rather than ones that merely nudge us to steer slightly around some new curves. In the security world, these forces (disruptions) come from three angles- business innovation, threat innovation, and efficiency innovation. The businesses we support are innovating for competitive advantage, as are the bad guys. For both of them, it's all about increasing the top line. The last category is more internal- efficiency innovation to increase the bottom line. Here's how I see the forces we're dealing with today, in no particular order:

1. Web browsers are inherently insecure. The very model of the world wide web is to pull different bits from different places, and render them all in a single view through the browser. Images from over here, text from over here, and, using iframes, entire sites from yet someplace else. It's a powerful tool, and I'm not criticizing this model; it just is what it is. From a security standpoint, this makes our life more than a little difficult. Even with a strictly enforced same origin policy, it's impossible to completely prevent cross-site issues, especially when people keep multiple sessions to multiple sites open all at the same time. That's why we have XSS, CSRF, and related attacks. We are trying to build a trust model where one end can never be fully trusted.
2. We have a massive repository of insecure code that grows daily. I'm not placing the blame on bad programmers; many of the current vulnerabilities weren't well understood when much of this code was written. Even today, some of these issues are complex and not always easy to remediate. We are also discovering new vulnerability classes on a regular basis, requiring review and remediation on any existing code. We're talking millions of applications, never mind many millions of lines of code. Even the coding frameworks and tools themselves have vulnerabilities, as we just saw with the latest Ruby issues.
3. The volume of sensitive data that's accessible online grows daily. The Internet and web applications are powerful business tools. It only makes sense that we connect more of our business operations online, and thus more of our sensitive data and business operations are Internet accessible.
4. The bad guys know technology. Just as it took time for us to learn and adopt new technologies, the bad guys had to get up to speed. That window is closed, and we have knowledgeable attackers.
5. The bad guys have an economic infrastructure. Not only can they steal things, but they have a market to convert the bits to bucks. Pure economics give them viable business models that depend on generating losses for us.
6. Bad guys attack us to steal or assets (information) or hijack them to use against others (e.g., to launch a big XSS attack). They also sometimes attack us just to destroy our assets, but not often (less economic incentive, even for DoS blackmail).
7. Current security tools are not oriented to the right attack vectors. Even WAFs offer limited effectiveness since they are more tied to our network security models than our data/information-centric models.
8. We do not have the resources to clean up all existing code, and we can't guarantee future code, even using a secure SDLC, won't be vulnerable. This is probably my most contentious assumption, but most of the clients I work with just don't have the resources to completely clean what they do have, and even the best programmers will still make mistakes that slip through to production.
9. Code scanning tools and vulnerability analysis tools can't catch everything, and can't eliminate all false positives. They'll never catch logic flaws, and even if we had a perfect tool, the second a new vulnerability appeared we'd have to go back and fix everything we'd built up to that point.
10. We're relying on more and more code and web services developed by others. From machine-generated web applications, to frameworks and off-the-shelf web apps we customize, to mashups where we directly pass content generated by someone else to our users.
11. "Web applications" is a misnomer- we mean the entire stack: web servers, web application servers, the databases behind them, and all the various interconnected n tiers. Many of these are internally accessible, creating an additional vector for attack.

To rephrase these a bit:

1. Bad guys are focused on our web applications, and are intelligent and motivated.
2. We have a lot of insecure code, keep generating more, and can't rely on secure development to fix it all.
3. WAFs and code scanning help, but aren't enough.
4. We need to protect a big, complex stack including content and services outside our control.

Following these forces, I'm drawing some assumptions about what any solution needs to look like:

1. We need to include browser elements, but can't trust the browser.
2. We need to monitor and enforce at the transaction level, both for audibility and for logic flaws and other security issues.
3. Such monitoring and enforcement needs to run from the browser to the database.
4. Any solution needs to understand the application and database, not just layer over it.
5. We need to filter anything we pass on to the user.
6. We need to focus on protecting the information.

I'm not totally thrilled with how I've laid this out, but I think it's reasonably understandable. Tomorrow I'll walk through how I think the technology will develop, and where today's tools fit in. I suspect Adrian will start chiming in, once he gets off the road, with his own interpretations.

–Rich

Crime, Communication, and Statistics

By Rich

'I'm not sure if it's the innate human desire to recognize patterns even when they don't exist, or if the stars really do align on occasion, but sometimes a series of random events hit at just the right time to inspire a little thought.

Or maybe I'm just fishing.

This week is an interesting one on the home front. It's slowly emerging that we're having some crime problems in the community. There has been a rash of vehicle break-ins and other light burglary. I found out about it when a board member of our HOA (and former cop) posted in our community forums that we've hired an off-duty Phoenix police officer to patrol our neighborhood, on top of the security company we already have here. We've got a big community center with a pool, so we need a little more security than the average subdivision.

Our community forums are starting to fill up with reports from throughout the community and I highly suspect this recent spree will be ending soon. All 900 homes now have access to suspect descriptions, targets, areas of concern, and so on. We're all locking up tighter and keeping our eyes open. Already some activity was caught on camera and turned over to the police. We know the bad guy's techniques, tactics, and operations. With this many eyeballs looking for them, the odds are low they'll be working around here much longer.

We've had problems for months, and the private security was ineffective. There is just too much territory for them to cover effectively. This spree could have potentially gone on forever, but now that the community is engaged we've moved from relying on 2 people to nearly 900 for our monitoring and defense.

We've taken the edge, just by sharing and talking.

In the security world some interesting tidbits have popped up this week. First came Debix with their fraud numbers, and now Verizon with their forensic investigation's breach report. On a private email list I was slightly critical of Verizon, but I realized I'm just being greedy and wanted more detail. While it could be better, this is some great information to get out there (thanks for making me take a second look, Hoff).

I shouldn't have been critical, because when it comes to data breaches we should be thankful for any moderately reliable stats we can get our hands on.

Between these two reports, a couple of things jumped out at me. First, I think these finally debunk all the insider threat marketing garbage. No one ever really had those numbers; trust me, since I saw my "estimate" from Gartner quoted as a hard number for years. This now aligns with my gut feeling, which is that there are more bad guys on the outside than the inside, although inside attacks can be more devastating under the right circumstances.

To further support this, the Verizon report also indicates that many attacks on the inside (or from partners) are really attacks from the outside that compromised an internal system. This supports my controversial positions on how we should treat the insider threat.

The second major point is that we rarely know where our data is, or if our systems are really configured correctly. Both of these are cited in the report as major sources of breaches- unknown data, unknown systems, and misconfigured systems. This is strongly supported by the root cause analysis work I've done on data breaches (in my data breach presentation; haven't written it in paper/blog form yet). People wonder why I'm such a big fan of DLP. Just think about how much risk you can reduce by scanning your environment for sensitive data in the wrong places.

FInally, it's clear that web applications are a huge problem. Verizon claims web apps were involved in 34% of cases. Again, this supports my conclusion from data breach analysis that links more fraud to application compromises than lost tapes or laptops. The Debix numbers also indicate no higher fraud levels for lost tapes than normal background levels of fraud.

We're on the early edge of building our own neighborhood watch. We're starting to see the first little nibs of hard breach data, and they're already defying conventional wisdom. By communicating more and sharing, we are better able to make informed risk and security decisions. Without this information, the bad guys can keep cruising our neighborhoods with impunity, stealing whatever we accidentally leave in our cars overnight.

–Rich

Web Application Security: We Need Web Application Firewalls To Work. Better.

By Rich

Jeremiah Grossman is just finishing up his keynote at the SANS conference on web application security. Jeremiah and I have talked a few times about the future of web application security, and we both agree that many current approaches just can't solve the problem. It's increasingly clear that no matter how good we are at secure programming (SDLC) , and no matter how effective our code scanning and vulnerability analysis tools are, neither approach can "solve" our web application security problem.

We not only develop code at a staggering pace, we have a massive legacy code base. While many leading organizations follow secure software development lifecycles, and many more will be adopting at least some level of code scanning over the next few years thanks to PCI 6.6, it's naive to think that even the majority of software will go through secure development any time soon. On top of that, we are constantly discovering new vulnerability classes that affect every bit of code written in the past. And, truth be told, no tool will ever catch everything, and even well-educated people still make mistakes.

Since these same issues affect non-web software, we've developed some reasonably effective ways to protect ourselves on that side. The key mantra is shield and patch. When we discover a new vulnerability, we (if possible) shield ourselves through firewalls and other perimeter techniques to buy us time to fix (patch) the underlying problem. No, it doesn't always work and we still have a heck of a lot of progress to make, but it is a fundamentally sound approach.

We're not really doing this much in the web application world. The web application firewall (WAF) market is growing, but has struggled for years. Even when WAFs are deployed, they still struggle to provide effective security. If you think about it, this is one big difference between a WAF and a traditional firewall or IPS. With old school vulnerabilities we know the details of the specific vulnerability and (usually) exploit mechanism. With WAFs, we are trying to block vulnerability classes instead of specific vulnerabilitie s . This is a HUGE difference. The WAF doesn't know the details of the application or any application-specific vulnerabilities, and thus is much more limited in what it can block.

I don't think stand-alone external WAFs will ever be effective enough to provide us the security we need for web applications. Rather, we need to change how we view WAFs. They can no longer be merely external boxes protecting against generic vulnerabilities; they need tighter integration into our applications. In the long term, I've branded this Application and Database Monitoring and Protection (ADMP) as we create a dedicated application and database security stack that links from the browser on the front end, to the DB server on the back.

There are a few companies exploring these new approaches today. Jeremiah's company, WhiteHat Security, has teamed up with F5 to provide specific vulnerability data from a web application to the F5 WAF. Fortify is moving down the anti-exploitation path with real-time blocking (and other actions) directly on the web application server. Imperva is tying together their WAF and database activity monitoring. (I'm sure there are more, but these are the web-application specific companies taking this path I can remember offhand). They are all taking different approaches, but all recognize that "static" WAFs or code scanning alone are pretty limited.

–Rich

Live at SANS WhatWorks: App Sec and Pen Testing

By Rich

I'm out in Vegas at the SANS WhatWorks Summits on application security and penetration testing. I like the format of these events, which mix a few expert talks with a whole slew of user panels. I've previously spoken at the DLP and Mobile Encryption Summits.

If you're in Vegas, drop me a line. Otherwise, stay tuned for some posts on these topics. One of the nice things about these events is there are actually power outlets for the audience; so between that and my EVDO card I can write live at the event.

Right now I'm sitting in Jeremiah Grossman's keynote session. His statistics on the probable number of 0-days in web applications are simply astounding. I've seen this content before, and it never ceases to stun me. More on that in a minute as I dedicate a post to how we need to change our perspective on web applications...

–Rich