Vulnerability Assessment

Verizon Has Most of the Web Application Security Pieces… But Do They Know It?

By Rich

Last week Verizon Business announced that they now offer web application vulnerability assessment software as a service. Specifically, they are reselling a full version of WhiteHat Security's offering, customized for Verizon business customers.

To be honest I'm somewhat biased here since WhiteHat's CTO, Jeremiah Grossman, is a friend; but I've been fairly impressed with their model of SaaS-based continuous web app vulnerability assessment using a combination of scanning and manual validation to reduce false positives. Jeremiah's marketing folks will hate it when I say this, but in my mind it's closer to penetration testing than the other SaaS vulnerability assessment products, which rely completely on automated scanning. Perhaps instead of calling this "penetration testing" we can call it "exploit validation". Web application vulnerabilities are tougher to deal with from a risk management perspective since, on the surface, it can be very difficult to tell if a vulnerability is exploitable; especially compared to the platform vulnerabilities typically checked by scanners. Since all web applications are custom, it's important to validate those vulnerabilities to determine overall risk, as the results of a blind scan are generally full of potential false positives -- unless it has been de-tuned so much that the false negative rate is extremely high instead.

Verizon Business also sells a managed web application firewall, which they mention in the press release. If you refer back to our Building a Web Application Security Program series and paper; vulnerability assessment, penetration testing, and web application firewalls are core technologies for the secure deployment and secure operations phases of managing web applications (plus monitoring, which is usually provided by the WAF and other logging).

In that series and paper, we also discussed the advantages of WAF + VA, where you dynamically generate WAF policies based on validated vulnerabilities in your application. This supports a rapid "shield then patch" model.

In the released information, Verizon mentions that they support WAF + VA. Since we know they are using WhiteHat, that means their back-end for WAF is likely Imperva or F5, based on WhiteHat's existing partnerships.

Thus Verizon has managed VA, managed WAF, managed WAF + VA, and some penetration testing support, via the VA product.

They also have a forensics investigation/breach response unit which collects all the information used to generate the Data Breach Investigations Report.

Let's add this up... VA + Exploit Validation (lightweight pen testing) + WAF + (WAF + VA) + incident response + threat intelligence (based on real incident responses). That's a serious chunk of managed web security available from a single service provider. My big question is: do they realize this? It isn't clear that they are positioning these as a combined service, or that the investigations/response guys are tied in to the operations side.

The big gap is anything in the secure development side... which, to be honest, is hard (or impossible) for any provider unless you outsource your actual development to them.

SecureWorks is another vendor in this space, offering web application assessments and managed WAF (but I don't know if they have WAF + VA)... and I'm pretty sure there are some others out there I'm missing.

What's the benefit? These are all pieces I believe work better when they can feed information to each other... whether internal or hosted externally. I expect the next pieces to add are better integrated application monitoring, and database activity monitoring.

(For the disclosure record, we have no current business relationships with WhiteHat, Verizon, F5, or SecureWorks, but we have done work with Imperva).

–Rich

Penetration Testing Market Update, Part 2

By Rich

This is part 2 of a series, click here for Part 1

Penetration testing solution and market changes

I'm not exactly sure when Core Security Technologies and Immunity started business, but before then there were no dedicated commercial penetration testing tools. There were a number of vulnerability scanners, and plenty of different "micro" tools to help with different parts of a pen test, but no dedicated exploitation tools. Metasploit also changed this on the non-commercial side. For those who aren't experts in this area, it's important to remember that a vulnerability assessment is not a penetration test -- vulnerability assessment determines if a system may be vulnerable to an attack, while penetration testing determines if that vulnerability is exploitable.

Update- Ivan from Core emailed that they started as consulting in 1996, and the first version of Impact was released in 2002.

Rather than repeating Nick Selby's excellent market summary of the three penetration testing tools providers over at IANS, I'll focus on the changes we're seeing in the overall market.

1. The market is still dominated by services, with quality ranging from excellent to absolute snake oil. Even using a tool like Core, by far the most user-friendly, you still need a certain skill level to perform a reasonable test.
2. The tools market is increasing, as Core and Immunity have experienced reasonable growth, with extensive growth of the Metasplit user community.
3. Partnerships between vulnerability assessment vendors and penetration testing solution providers have grown. This was pretty much completely driven by Core until the Metasploit acquisition by Rapid7. Core partners with Tenable, Qualys, nCircle, IBM, Lumension, GFI, and eEye. Update- Immunity partners with Tenable, I missed that in my initial research.
4. Web application vulnerability assessment tools (and services) almost always include some level of penetration-testing capabilities. This is a technology requirement for effective results, since it is extremely difficult to accurately validate many web application vulnerability types without some degree of exploitation. VA tools tend to restrict themselves to prevent damaging the application being tested, and (as with nearly any vulnerability assessment), can normally be run against non-production targets with less safety, in order to produce deeper and more accurate results.
5. Any penetration test worth its salt includes web applications within the scope, and pen testing tools are increasing their support for web application testing.

I expect to see greater blurring of the lines between vulnerability assessment and penetration testing in the web application area, which will spill over into the infrastructure assessment space. We'll also see increasing demand for internal penetration testing, especially for web applications.

Core will increase its partnerships and integration on the VA side, and could see an acquisition if larger VA vendors (a small list) see growing customer demand for penetration testing -- which I do not expect in the short term. The VA market is larger and if those vendors see pen testing client demands, or greater competition from Rapid7, they can leverage their Core partnerships. Core's Impact Essential tool is the first to target individuals who aren't full-time security professionals or penetration testers, and run on an automated schedule. While it doesn't have nearly the depth of the Pro product, it could be interesting for continuous testing. The real question is whether customers perceive it as either reducing their process costs for vulnerability management (via prioritization and elimination of non-exploitable vulnerabilities), or a replacement for an existing VA solution. If Impact Essential can't be used to cut overall costs, it will be hard to justify in the current economic environment.

As Nick concluded, Immunity will need to improve their UI to increase adoption beyond organic growth... unless they plan to stay focused on dedicated penetration testers. They should also consider some VA partnerships, as they will be the only penetration testing tool not partnered or integrated with VA Update- I was incorrect, Immunity also partners with Tenable. Apologies for missing that in my initial research.. I agree with Nick: Immunity is most at risk in the short term from the Metasploit commercialization. If the UI improves, Immunity could use cost to compete, and some VA vendors might add them as an additional partner.

Rapid7 just jumped from being one of the less-known VA players to a household name for anyone who pays attention to penetration testing. This is a huge opportunity, but not without risks. Metasploit is an awesome tool (I've used it since version 1... in the lab), but not yet enterprise class. The speed, usefulness, and usability of its integration will play a major role in its long-term success and ability to springboard off the large amount of press and additional name recognition associated with this acquisition. H D also needs to aggressively maintain the Metasploit community, or Rapid7 will lose a large fraction of Metasploit's value and have to pay staff to replace those volunteers. Quality assurance, of the product as well as the exploits, will also be important to maintain; this could reduce the speed of releasing exploits which Metasploit is famous for.

Rapid7 also faces risks due to Metasploit's BSD license. There is nothing to prevent any other vendor from taking and using the code base. This is a common risk when commercializing any free/open source software, and we've seen both successes and failures.

Conclusion

Here's how I see things developing:

1. For infrastructure/non-web applications we will see growing demand for exploit testing automation. The vulnerability assessment vendors will add native capabilities, and Core (and Immunity, if they choose) will add more native VA capabilities and find themselves competing more with VA vendors. My gut feel is that VA vendors (other than Rapid7) will only add the most basic of capabilities, leaving the pen testing vendors with a technical advantage until both markets completely merge. That might not matter to most organizations, which either won't understand the technology differentiation, or won't care.
2. There will continue to be a need for in-depth tools to support professional penetration testers. This market will continue to grow, but will not offer the opportunities of the broader, 'lights-out' automated side of the market.
3. Overall, the penetration testing tools market will continue to grow. This acquisition and other market trends validate the usefulness of this market, especially in assisting with remediation prioritization -- not just problem identification.
4. The greatest area of growth will be in web applications, and as I mentioned before the lines between pure VA and pure penetration testing will completely blur in this area.
5. All the penetration testing vendors will benefit from the Metasploit acquisition. Immunity faces the greatest mid-term risk, Core the greatest potential for price pressure, and Rapid7 the risk of losing the Metasploit community and seeing their work appear in competing products. All three vendors can manage potential risks, but the answers aren't necessarily easy.

A bit of disclosure -- I haven't been briefed formally by Immunity, so I could be missing part of their strategy. Although I've talked with most of the VA vendors, I haven't specifically discussed their plans for exploit validation or penetration testing, and I base my conclusions more on conversations with end users.

–Rich

Penetration Testing Market Grows and Matures, but Faces Challenges

By Rich

With last week's acquisition of Metasploit by Rapid7, I thought it might be a good time to do a review of the penetration testing market and the evolving role of pen testing in the security arsenal. We've seen a few different shifts over the past few years in how organizations use pen testing, and I believe this acquisition -- combined with changes in enterprise infrastructure -- indicates that pen testing is becoming more essential, more closely tied to vulnerability assessment, and generally more mature.

First, a bit of a disclaimer: I'm approaching this as an analyst, not a penetration tester. Although I've used many of the tools in demonstrations and the lab, I've never worked as a pen tester and don't claim to have that skill set. I'm fairly sure my BBS hacking experience from the mid-80's doesn't really count.

There are two important issues we need to focus on when evaluating penetration testing -- changes in need and value, and changes in delivery methods and tools.

The value of penetration testing

There is sometimes a debate on the value of penetration testing. Some question its usefulness, since a test by a competent practitioner is pretty much guaranteed to succeed, but highly unlikely to find every exploit path into the organization. More comprehensive tests will find more holes, but at a much higher cost. In some verticals (particularly financials and some types of government organizations) the risk is so high that this is an accepted cost, but for less-aware and less-targeted verticals, or small and mid-sized organizations, a basic vulnerability or program assessment can find more issues at lower cost.

That's because, until fairly recently, penetration testing was dominated by external service organizations performing broad network and host based assessments. Tests were used to:

1. Scare management into spending more on security.
2. Get a general sense of how hardened the organization was.
3. Find and fix any obvious holes that might stand out either in an untargeted scan/attack, or to an attacker willing to spend a little more time with limited resources.

Basically, a pen test would give you a good sense of how you'd withstand an attack by an opponent at the same skill level as your testing team, for the amount of time/effort you were willing to pay for. Obviously there are a lot of exceptions, and I'm only talking about general market trends. But at this stage, unless you were a big target, a vulnerability assessment (including an internal assessment) would provide sufficient value at a lower cost.

That's still how many tests are used, but we've seen a shift in the past few years due to a few changes in the risk and threat landscape. Specifically:

1. An increase in highly targeted attacks.
2. Greater use of web applications, and more web application attacks (one of the single biggest source of losses in recent major reported incidents).
3. A market and economic system for taking advantage of exploited data.
4. Evolution of technologies & vulnerabilities, coupled with much shorter exploit creation/adoption cycles than in the past. For example, zero day attacks were extremely uncommon just 2-3 years ago, but now seem to appear monthly.

The bad guys are making serious money, are going after harder targets, and are taking advantage of our rapid adoption of web technologies. They really have to, since we've gotten a lot better at securing our networks and endpoints (yes, we really have, from an overall trends standpoint).

These factors change the focus and requirements for penetration testing. While this is merely one analyst's opinion, and some of these are very early trends, here's what I'm seeing:

1. Organizations are increasing the frequency of vulnerability assessments and penetration testing, to reduce between-assessment risks. In some cases these are continuous programs.
2. Penetration tests are being more closely tied to vulnerability assessments in order to determine risk and prioritize patches and other defenses.
3. The line between a vulnerability assessment and a penetration test is almost completely blurred for web applications -- especially custom web applications.
4. There is greater use of, and need for, penetration testing during development and pre-production phases, since some testing is prohibitively risky on a production system.

Penetration testing is being more closely tied to vulnerability assessment on non-web systems to help prioritize. A VA doesn't necessarily tell you how exploitable a target is, and it certainly won't tell you what the bad guy can potentially gain. A penetration test helps validate the overall risk and determine the potential impact and losses (not in financial terms -- that's for another day). A vulnerability scan can tell you that system X is vulnerable to attack Y, but you often need to go a step further with a pen test to determine if data Z is at risk. This is especially true for web applications, but also important for other types of assets.

The overall focus is shifting away from "Can someone break in, and how long will it take them?" to "Where are we most exposed, and what are our potential losses?" Penetration testing is becoming more of a prioritization and secure development tool.

See part 2 for how these factors change the solutions and penetration testing market

–Rich

Rapid7 Acquires Metasploit

By Adrian Lane

Rapid7 acquires Metasploit, the open source penetration testing platform. Wow. All I can say is 'Wow'. I had been hearing rumors that Rapid7 was going to make an acquisition for weeks, but this was a surprise to both Rich and myself. Still coming to terms with what it means, and I have no clue what the financial terms look like, but almost certainly this is a cash+stock deal. On the surface, it is a very smart move for Rapid7.

Metasploit is considerably better known than Rapid7. Metasploit is a fixture in the security research world and there are far more people using Metasploit than Rapid7 has customers. If nothing else, this gets Rapid7 products in the hands of the people who are shaping web application security, and defining how penetration testing and vulnerability management will be conducted. In a quickly evolving market like pen testing, access to that community is invaluable for a commercial vendor. Plus they get H D Moore on staff, which is a huge benefit.

Metasploit is a well-architected framework that provides for easy extensibility and can be customized in innumerable ways. If you want to test anything from smart phones to databases, this platform will do it, from targeted exploits to fuzzing. Sure, there is work on your part and accessibility to people other than security researchers is low compared to commercial products like Core Security's Impact, but it's a solid platform and the integration of the two should not be difficult. It's more a question of how best to allow Metasploit to continue its open source evolution while leveraging scans into meaningful vulnerability chaining, as well as risk scoring.

Neither is exactly an 'enterprise ready' product. That's not a slam, as NeXpose performs its primary function as well as most. But Rapid7's platform is just now breaking ground into larger companies. They have a long way to go in UI, ease of use, pragmatic analysis, integration of risk scoring, SaaS, exploit chaining, and back-end integration. That said, I am not sure they need to be an enterprise ready product, at least in the short term. It makes more sense to continue their mid-market penetration while they complete the integration. Breadth of function, which is what they now have, has proven to be a major factor in winning deals over the last couple years. They can worry about the advanced non-technical stuff later.

Identity in the market is an issue for Rapid7. They have waffled between general assessment, pen testing, and vulnerability management, without a clear identity or differentiator when going toe-to-toe with Qualys, nCircle, Tenable, Secunia, and the like. Sure, 'compliance scoring' is a useful marketing gimmick, but Metasploit gives them a unique identity and differentiation. Rather than scan-and-patch for known vulnerabilities, focusing mostly inside the network, they will now be able to go far deeper into externally facing custom applications. Taking a risk score across multiple applications and/or platforms is a better approach. If the two platforms are properly integrated, they'll be useful to IT, security, and software development.

I am sure Rich will chime in with his own take later in the week. Wow.

–Adrian Lane

Database Assessment Solutions, Part 4: Vulnerability and Security Policies

By Adrian Lane

Understanding and Choosing a Database Assessment Solution, Part 4: Vulnerability and Security Policies

I was always fascinated by the Sapphire/Slammer worm. The simplicity of the attack and how quickly it spread were astounding. Sure, it didn't have a malicious payload, but the simple fact that it could have created quite a bit of panic. This event is what I consider the dawn of database vulnerability assessment tools. From that point on it seemed like every couple of weeks we were learning of new database vulnerabilities on every platform. Compliance may drive today's assessment purchase, but the vulnerabilities are always what grabs the media's attention, and it remains a key feature for any database security product.

Prior to writing this post I went back and looked at all the buffer overflow and SQL injection attacks on DB2, Oracle, and SQL Server. It struck me when looking at them -- especially those on SQL Server -- why half of the administrative functions had vulnerabilities: whoever wrote them assumed that the functions were inaccessible to anyone who was not a DBA. The functions were conceptually supposed to be gated by access control and therefore safe. It was not so much that the programmers were not thinking about security, but they made incorrect assumptions about how the database internals like the parser and preprocessor worked. I have always said that SQL injection is an attack on the database through an application. It's true, but technically the attacks are also getting through internal database processing layers prior to the exploit, as well as an eternal application layer. Looking back at the details it just seemed reasonable we would have these vulnerabilities, given the complexity of the database platforms and the lack of security training among software developers. Anyway, enough rambling about database security history.

Understanding database vulnerabilities and knowing how to remediate -- whether through patches, workarounds, or third party detection tools -- requires significant skill and training. Policy research is expensive, and so is writing and testing these policies. In my experience over the four years that I helped define and build database assessment policies, it would take an average of 3 days to construct a policy after a vulnerability was understood: A day to write and optimize the SQL test case, a day to create the description and put together remediation information, and another day to test on supported platforms. Multiply by 10 policies across 6 different platforms and you get an idea of the cost involved. Policy development requires a full-time team of skilled practitioners to manage and update vulnerability and security policies across the half dozen platforms commonly supported by the vendors. This is not a reasonable burden for non-security vendors to take on, so if database security is an issue, don't try to do this in-house! Buying an aftermarket product excuses your organization from developing these checks, protecting you from specific threats hackers are likely to deploy, as well as more generic security threats.

What specific vulnerability checks should be present in your database assessment product? In a practical sense, it does not matter. Specific vulnerabilities come and go too fast for any list to be relevant. What I am going to do is provide a list of general security checks that should be present, and list the classes of vulnerabilities any product you evaluate should have policies for. Then I will cover other relevant buying criteria to consider.

General Database Security Policies

• List database administrator accounts and how they map to domain users.
• Product version (security patch level)
• List users with admin/special privileges
• List users with access to sensitive columns or data (credit cards, passwords)
• List users with access to system tables
• Database access audit (failed logins)
• Authentication method (domain, database, mixed)
• List locked accounts
• Listener / SQL Agent / UDP, network configuration (passwords in clear text, ports, use of named pipes)
• Systems tables (subset) not updatable
• Ownership chains
• Database links
• Sample Databases (Northwind, pubs, scott/tiger)
• Remote systems and data sources (remote trust relationships)

Vulnerability Classes

• Default Passwords
• Weak/blank/same as login passwords
• Public roles or guest accounts to anything
• External procedures (CmdExec, xp_cmdshell, active scripting, exproc, or any programatic access to OS level code)
• Buffer overflow conditions (XP, admin functions, Slammer/Sapphire, HEAP, etc. -- too numerous to list)
• SQL Injection (1=1, most admin functions, temporary stored procedures, database name as code -- too numerous to list)
• Network (Connection reuse, man in the middle, named pipe hijacking)
• Authentication escalation (XStatus / XP / SP, exploiting batch jobs, DTS leakage, remote access trust)
• Task injection (Webtasks, sp_xxx, MSDE service, reconfiguration)
• Registry access (SQL Server)
• DoS (named pipes, malformed requests, IN clause, memory leaks, page locks creating deadlocks)

There are many more. It is really important to understand that the total number of in policies any given product is irrelevant. As an example, let's assume that your database has two modules with buffer overflow vulnerabilities, and each has eight different ways to exploit it. Comparing two assessment products, one might have 16 policies checking for each exploit, and the other could have two policies checking for two vulnerabilities. These products are functionally equivalent, but one vendor touts an order of magnitude more policies, which have no actual benefit. Do NOT let the number of policies influence your buying decision and don't get bogged down in what I call a "policy escalation war". You need to compare functional equivalence and realize that if one product can check for more vulnerabilities in fewer queries, it runs faster! It may take a little work on your part to comb through the policies to make sure what you need is present, but you need to perform that inspection regardless.

You will want to carefully confirm that the assessment platform covers the database versions you have. And just because your company supposedly migrated to Oracle 11 some time back does not mean you get to discount Oracle 9 database support, because odds are better than even that you have at least one still hanging around. Or you don't officially support SQL Server, but it just so happens that some of the applications you run have it embedded. Furthermore, mergers and acquisitions bring unexpected benefits, such as database platforms you did not previously have in house. Or plans to migrate off a current platform have a way of changing suddenly, with the database sticking around in your organization many years longer than anticipated. Broad database coverage should weigh heavily in your buying decision.

The currency of policies (at least for coverage of the latest vulnerabilities) is very important. Check to make sure the vendor has a solid track record of delivering policy updates no less than once a quarter. The database vendors typically release a security patch each quarter, so your assessment vendor should as well. A 'plan' to do so is insufficient and should be a warning signal. Press vendors for proof of delivery, such as release documentation or policy maintenance update announcements to demonstrate consistent delivery of updated policies vulnerability polices.

Cross reference policies with the database vendor information. One of the interesting friction points between the database vendors and the vulnerability scanning vendors is the production of complete and detailed information on vulnerabilities. You need to have a clear and detailed explanation of each vulnerability to understand how a vulnerability affects your organization and what workarounds may be at your disposal. While assessment vendors are motivated to provide detailed information on the vulnerability itself, for whatever reason the database vendors tend to offer terse descriptions of the threats and corresponding patch data. Press the assessment vendor for detailed information, but keep in mind the database vendor must be considered the primary source of complete and accurate remediation information. It is wise, either during an evaluation or in production, to cross reference the information provided, and weigh how well each policy documents each threat.

Finally, database security advice comes from many different sources. The database vendors usually supply best practice checklists for free. The assessment products usually list the policies that they have developed over time from what they have learned in the field. There are other independent blogs, such as Pete Finnigan's, that offer solid advice. Finally, most database vendors have regional user groups that share information on how to approach database security, which I have always found useful. Check to see if your assessment vendor has what you need, and they probably will given that the major data breaches as of this writing are leveraging the basic vulnerabilities. If you find something is missing, find out if your vendor can provide it for you. We will get into policy customization in more detail in the next post, as well as cover integration and policy set management topics.

This was supposed to be a short post on vulnerability and security best practices. Short because these two topics are not good indicators of how well a particular database assessment product will meet your needs. It may be interesting to a researcher like myself, but I realize this might be more information than you need or want. Data collection options discussed in part 3, alongside operations and compliance policies which we will discuss in part 5, have a greater bearing on how useful the product will be.

–Adrian Lane

Understanding and Choosing a Database Assessment Solution, Part 1: Introduction

By Adrian Lane, Rich

Last week I provided some advice regarding database security to a friend's company, which who is starting a database security program. Based on the business requirements they provided, I made several recommendations on products and processes they need to consider to secure their repositories. As some of my answers were not what they expected, I had to provide a lot of detailed analysis of why I provided the answers I did. At the end of the discussion I began asking some questions about their research and how they had formed some of their opinions. It turns out they are a customer of some of the larger research firms and they had been combing the research libraries on database security. These white papers formed the basis for their database security program and identified the technologies they would consider. They allowed me to look at one of the white papers that was most influential in forming their opinions, and I immediately saw why we had a disconnect in our viewpoints.

The white paper was written by two analysts I both know and respect. While I have some nit-picks about the content, all in all it was informative and a fairly good overview document ... with one glaring exception: There was no mention of vulnerability assessment! This is a serious omission as assessment is one of the core technologies for database security. Since I had placed considerable focus on assessment for configuration and vulnerabilities in our discussion, and this was at odds with the customer's understanding based upon the paper, we rehashed a lot of the issues of preventative vs. detective security, and why assessment is a lot more than just looking for missing database patches.

Don't get me wrong. I am a major advocate and fan of several different database security tools, most notably database activity monitoring. DAM is a very powerful technology with a myriad of uses for security and compliance. My previous firm, as well as a couple of our competitors, were in such a hurry to offer this trend-setting, segment-altering technology that we under-funded assessment R&D for several years. But make no mistake, if you implement a database security program, assessment is a must-have component of that effort, and most likely your starting point for the entire process. When I was on the vendor side, a full 60% of the technical requirements customers provided us in RFP/RFI submission requests were addressed through assessment technology! Forget DAM, encryption, obfuscation, access & authorization, label security, input validation, and other technologies. The majority of requirements were fulfilled by decidedly non-sexy assessment technologies. And with good reason. Few people understand the internal complexities of database systems. So as long as the database ran trouble-free, database administrators enjoyed the luxury of implicit trust that the systems under their control were secure. Attackers demonstrate how easy it is to exploit un-patched systems, gain access to accounts with default passwords, and leverage administrative components to steal data. Database security cannot be assumed, but it must be verified. The problem is that security teams and internal auditors lack the technical skills to query database internals; this makes database assessment tools mandatory for automation of complex tasks, analysis of obscure settings, and separation of duties between audit and administrative roles.

Keep in mind that we are not talking about network or OS level inspection -- rather we are talking about database assessment, which is decidedly different. Assessment technologies for database platforms have continued to evolve and are completely differentiated from OS and network level scans, and must be evaluated under a different set of requirements than those other solutions. And as relational database platforms have multiple communication gateways, a complete access control and authorization scheme, and potentially multiple databases and database schemas all within a single installation, the sheer complexity requires more than a cursory inspection of patch levels and default passwords. I am defining database assessment as the following:

Database Assessment is the analysis of database configuration, patch status, and security settings; it is performed by examining the database system both internally and externally -- in relation to known threats, industry best practices, and IT operations guidelines.

Because database assessment is continually under-covered in the media and analyst community, and because assessment is one of the core building blocks to the Securosis database security program, I figured this was a good time for the official kick-off of our blog series on Understanding and Selecting a Database Vulnerability Assessment Solution. In this series we will cover:

• Configuration data collection options
• Security & vulnerability analysis
• Operational best practices
• Policy management and remediation
• Security & compliance reporting
• Integration & advanced features

I will also cover some of the evolutions in database platform technology and how assessment technologies must adapt to meet new challenges. As always, if you feel we are off the mark or missing something, tell us. Reader comments and critiques are encouraged, and if they alter or research position, we credit commentors in any research papers we produce. We have comment moderation turned on to address blog spambots, so your comment will not be immediately viewable, but Rich and I are pretty good about getting comments published during business hours.

–Adrian Lane, Rich

Building a Web Application Security Program, Part 8: Putting It All Together

By Adrian Lane

'Whew! This is our final post in this series on Building a Web Application Security Program (Part 1, Part 2, Part 3, Part 4, Part 5, Part 6, Part 7), and it's time to put all the pieces together. Here are our guidelines for designing a program that meets the needs of your particular organization. Web application security is not a "one size fits all" problem. The risks, size, and complexity of the applications differ, the level of security awareness among team members varies, and most importantly the goals of each organization are different.

In order to offer practical advice, we needed to approach program development in terms of typical goals. We picked three use cases to represent common challenges organizations face with web app security, and will address those use cases with appropriate program models. We discuss a mid-sized firm tackling a compliance mandate for the first time, a large enterprise looking to improve security across customer-facing applications, and a mid-to-large organization dealing with security for internal applications. Each perspective has its own drivers and assumptions, and in each scenario different security measures are already in place, so the direction of each program will be different. Since we've been posting this over a series of weeks, before you dig in to this post we recommend you review Part 4: The Web Application Security Lifecycle which talks about all tools in all phases. First we describe the environment for each case, then overall strategy and specific recommendations.

Large Enterprise with Customer Facing Web Applications

For our first scenario, let's consider a large enterprise with multiple customer-facing web applications. These applications evolved to offer core business functions and are a principal contact point with customers, employees, and business partners. Primary business drivers for security are fraud reduction, regulatory compliance, and service reliability as tangible incentives. Secondary factors are breach preparedness, reputation preservation, and asset protection secondary -- all considerations for security spending. The question is not whether these applications need to be secured, but how. Most enterprises have a body of code with questionable security, and let's be totally honest here- these issues are flaws in your code. No single off-the-shelf product is going to magically make your application secure, so you invest not only in third-party security products, but also in improvements to your own development process which improve the product with each new release.

We assume our fictitious enterprise has an existing security program and the development team has some degree of maturity in their understanding of security issues, but how best to address problems is up for debate. The company will already have a 'security guy' in place, and while security is this guy's or gal's job, the development organization is not tasked with security assessments and problem identification. Your typical CISO comes from a network security background, lacks a secure code development background, and is not part of this effort. We find their security program includes vulnerability assessment tools, and they have conducted a review of the code for typical SQL injection and buffer overflow attacks. Overall, security is a combination of a couple third-party products and the security guy pointing out security flaws which are patched in upcoming release cycles.

Recommendations: The strategy is to include security within the basic development process, shifting the investment from external products to internal products and employee training. Tools are selected and purchased to address particular deficiencies in team skill or organizational processes. Some external products are retained to shield applications during patching efforts.

Training, Education, and Process Improvements: The area where we expect to see the most improvement is the skill and awareness of the web application development team. OWASP's top flaws and other sources point out issues that can be addressed by proper coding and testing ... provided the team knows what to look for. Training helps staff find errors and problems during code review, and iteratively reduces flaws through the development cycle. The development staff can focus on software security and not rely on one or two individuals for security analysis.

Secure SDLC: Knowing what to do is one thing, but actually doing it is something else. There must be an incentive or requirement for development to code security into the product, assurance to test for compliance, and product management to set the standards and requirements. Otherwise security issues get pushed to the side while features and functions are implemented. Security needs to be part of the product specification, and each phase of the development process should provide verification that the specification is being met through assurance testing. This means building security testing into the development process and QA test scenarios, as well as re-testing released code. Trained development staff can provide code analysis and develop test scripts for verification, but additional tools to automate and support these efforts are necessary, as we will discuss below.

Heritage Applications: Have a plan to address legacy code. One of the more daunting aspects for the enterprise is how to address existing code, which is likely to have security problems. There are several possible approaches for addressing this, but the basic steps are 1) identification of problems in the code, 2) prioritization on what to fix, and 3) planning how to fix individual issues. Common methods of addressing vulnerabilities include 1) rewriting segments of code, 2) method encapsulation, 3) temporary shielding by WAF ("secure & patch"), 4) moving SQL processing & validation into databases, 5) discontinuing use of insecure features, and 6) introduction of validation code within the execution path. We recommend static source code analysis or dynamic program analysis tools for the initial identification step. These tools are cost-effective and suitable for scanning large bodies of code to locate common risks and programming errors. They detect and prioritize issues, and reduce human error associated with tedious manual scanning by internal or external parties. Analysis tools also help educate staff about issues with certain languages and common programming patterns. The resulting arguments over what to do with 16k insecure occurrences of IFRAME are never fun, but acceptance of the problem is necessary before it can be effectively addressed.

External Validation: Periodic external review, through vulnerability assessment, penetration testing or source code review, is highly recommended . Skilled unbiased professionals with experience in threat analysis often catch items items which slip by internal scans, and can help educate development staff on different threat vectors. Plan on external penetration testing on a quarterly or biannual basis- their specific expertise and training goes far beyond the basics threats, and trained humans monitoring the output of sophisticated tools are very useful for detecting weaknesses that a hacker could exploit. We recommend the use of static testing tools for internal testing of code during the QA sweep, with internal penetration testing just prior to deployment so they can fully stress the application without fear of corrupting the production environment. Major releases should also undergo an external penetration test and review before deployment.

Blocking: This is one area that will really depend upon the specifics of your organization. In the enterprise use case, plan on using a Web Application Firewall. They provide basic protection and give staff a chance to remove security issues from the application. You may find that your code base is small and stable enough that you do not need WAF for protection, but for larger organizations this is not an option. Development and patching cycles are too long and cumbersome to counter threats in a reasonable timeframe. We recommend WAF + VA because in combination, they can relieve your organization from much of the threat research and policy development work for firewall rules. If your staff has the skill and time to develop WAF policies specific to your organization, you get customized policies at slightly greater expense in development costs. WAF isn't cheap, so we don't take this recommendation lightly, but it provides a great deal of flexibility in how and when threats are dealt with, today and as new threats evolve.

We recommend you take steps to improve security in every part of the development process. We are focused on improvements to the initial phases of development, as the impact of effort is greatest here, but we also recommend at the very least external assistance, and if budget allows, blocking. These later recommendations fill in other areas that need coverage, with penetration testing and web application firewalls. The risks to the enterprise are greater, the issues to overcome are more complex, and the corresponding security investment will therefore be larger. This workflow process should be formally documented for each stage of an application's lifecycle- from development through ongoing maintenance- with checkpoints for major milestones. Security shouldn't, and can't, be responsible for each stage, but should carry responsibility for managing the program and making sure the proper process is followed and maintained.

Mid-sized firm and PCI Compliance

If we are discussing web application security and compliance, odds are we are talking about the Payment Card Industry's Data Security Standard (PCI-DSS). No other compliance standard specifies steps to secure web applications like the PCI standard does. We can grouse about ambiguities and ways that it could be improved, but PCI is clearly the most widespread driver for web application security today, which is why our second use case is a mid-sized firm that needs to secure its web applications to satisfy PCI-DSS.

The profile for our company is a firm that generates a large portion of their revenue through Internet sales, and recent growth has made them a Tier 3 merchant. The commerce web site is relatively new (< 3 years) and the development team is small and not trained in security. Understanding the nuances of how criminals approach breaking code is not part of the team's skill set. PCI compliance is the mandate, and the team knows that they are both missing the basic requirements and susceptible to some kinds of attacks. The good news is that the body of code is small, and the web application accounts for a significant portion of the company's revenue, so management is supporting the effort.

In a nutshell, the PCI Data Security Standard is a security program specifically for companies that process credit card transactions for Internet commerce. In terms of compliance regulations, PCI-DSS requirements are clearer than most, making specific requirements for security tools and processes around credit card data. However, a company may also it has satisfy the spirit of the requirements in an alternate way, if it can demonstrate that the concern has been addressed. We will focus on the requirements outlined in sections 6.6 & 11.3, but will refer to sections 10 and compensating controls as well.

Recommendations: Our strategy focuses on education and process modifications to bring security into the development lifecycle. Additionally, we suggest assessment or penetration testing services to quickly identify areas of concern. Deploy WAF to address the PCI requirement immediately. Focus on the requirements to start, but plan for a more general program, and use compensating controls as your organization evolves. Use outside help and education to address immediate gaps, both in PCI compliance and more general application security.

Training, Education, and Process Improvements: Once again, we are hammering on education and training for the development team, including project management and quality assurance. While it takes time to come up to speed, awareness by developers helps keep security issues out of the code, and is cost-effective for securing the applications. Altering the process to accommodate fixing the code is essentially free, and code improvements become part of day to day efforts. With a small code base, education and training are easy ways to reap significant benefits as the company and code base grow.

External Help: Make friends with an auditor, or hire one as a consultant to help prepare and navigate the standard. While this is not a specific recommendation for any single requirement in PCI; auditors provide an expert perspective, help address some of the ambiguity in the standard, and assist in strategy and trade-off evaluations to avoid costly missteps.

Section 11.3.2: Section 11.3 mandates penetration testing of the network and the web application. In this case we recommend external penetration testing as an independent examination of the code. It is easy to recommend penetration testing, and not because it is required in the DSS specification, rather the independent & expert review of your application behavior will closely mimic the approach hackers will take. We also anticipate budget will require you make a choice between WAF and code reviews in section 6.6, so this will provide the necessary coverage. Should you use source code reviews, one could argue that acts as a compensating control for this section, but our recommendation is to stick with the external penetration testing. External testers provide much more than just a list of specific flaws, but also identify risky or questionable application behaviors in a near production environment.

Section 6.6: Our biggest debate internally was whether to recommend Web Application Firewall or expert code review to address section 6.6 of the PCI specification. The PCI Security Standards Council recommends that you do both, but it is widely recognized that this is prohibitively expensive. WAF provides a way to quickly meet the letter of 6.6's requirement, if not in spirit, provides basic monitoring, and is a flexible platform to block future attacks. The counter-arguments are significant and include cost, work required to customize policies for the application, and false positives & negatives. Alternatively, a code review by qualified security experts can identify weaknesses in application design and code usage and assist in education of the development team by pointing out specific flaws. Outside review is a very quick way to assess where you are and what you need. Down sides of review include cost, time to identify and fix errors, and that a constantly changing code base presents a moving target and thus requires repeated examinations.

Our recommendation here is deploy a WAF solution. Engaging a team of security professionals to review the code is an effective way to identify issues, but much of the value overlaps with the requirement of Section 11.3.2, periodic penetration testing of the application. The time to fix identified issues (even with a small-to-average body of code), with a development organization which is just coming to terms with security issues, is too long to meet PCI requirements in a timely fashion. Note that this recommendation is specific to this particular fictitious case- in other PCI audit scenarios, with a more experienced staff or a better handle on code quality, we might have made a different recommendation.

Monitoring: Database Activity Monitoring (DAM) is a good choice for Section 10 compliance- specifically by monitoring all access to credit card data. Web applications use a relational database back end to store credit card numbers, transactions, and card related data. DAM products that capture all network and console activity on the database platform provide a focused and cost-effective audit for all access to cardholder data. Consider this option for providing an audit trail for auditors and security personnel.

Internal Web Application Development

Our last use case is an internal web applications that serves employees and partners within a mid-to-large business. While this may not sound like a serious problem, given that companies have on average 1 internal application (web and traditional client/server) per 100 employees, even mid-sized companies have incredible exposure in this area. Using data from workflow, HR, accounting, business intelligence, sales, and other critical IT systems, these internal applications support employees and partners alike. And with access to pretty much all of the data within the company, security and integrity are major concerns. A common assumption is that these systems, behind the perimeter firewall, are not exposed to the same types of attacks as typical web applications, but this assumption has proven disastrous in many cases.

Investment here is motivated by fraud reduction, breach remediation, and avoidance of notification costs- and possibly by compliance. You may find this is difficult to sell to executive management if there is not a compliance mandate and hasn't been a previous breach, but if basic perimeter security is breached these applications need some degree of resiliency rather than blind confidence in network security and access control. TJ Maxx (http://www.tjx.com/) is an excellent illustration of the danger.

Strategy: Determine basic security of the internal application, fix serious issues, and leverage education, training, and process improvements to steadily improve the quality of code. We will assume that budgeting for security in this context is far smaller than for external-facing systems, so look to cooperate between groups and leverage tools and experience.

Vulnerability Assessment and Penetration Testing: Scanning web applications for significant security, patch and configuration issues is a recommended first step in determining if there are glaring issues. Assessment tools are a cost-effective way to establish baseline security and ensure adherence to minimum best practices. Internal penetration testing will help determine the overall potential risk and prioritization, but be extremely cautious of testing live applications.

Training, Education, and Process Improvements: These may be even more important in this scenario than in our other use cases, where the business justification provides incentive to invest in security, internal web applications may not get the same degree of attention. For these applications that have a captive audience, developers have greater controls over the types of environments that they support what can be required in terms of authentication. Use these freedoms to your advantage. Training should focus on common vulnerabilities within the application stack that is being used, and give critical errors the same attention that top priority bugs would receive. Verify that these issues are tested, either as part of the VA sweep, or a a component of regression testing.

Monitoring: Monitoring for suspicious activity and system misuse is a cost-effective way to detect issues and react to them. We find WAF solutions are often too expensive for deployment across hundreds of internal applications distributed across a company, and a more cost-effective approach to collecting and analyzing activity is highly recommended. Monitoring software that plugs into the web application is often very effective for delivering some intelligence at low cost, but the burden of analyzing the data then falls on development team members. Database Activity Monitoring can effectively focus on critical information at the back end and is more mature than Web Application Monitoring.

This segment of the series took much longer to write than we originally anticipated, as our research gave us conflicting answers to some questions, making our choices were far from easy. Our recommendations really depend upon the specifics of the situation and the organization. We approached this with use cases to demonstrate how the motivating factors; combined with the current state of web security; really guides the selection of tools, services, and process changes.

We found in every case that security as part of the overall development process is the most cost-effective and least disruptive to normal operations, and is our principal recommendation for each scenario. However, as transformation of a web application does not happen overnight, we rarely have the luxury of waiting for the development team to address all security issues is not realistic; in the meantime, external third-party services and products are invaluable for dealing with immediate security challenges.

–Adrian Lane

Building a Web Application Security Program: Part 6, Secure Deployment

By Rich

In our last episode, we continued our series on building a web application security program by looking at the secure development stage (see also Part 1, Part 2, Part 3, Part 4, and Part 5).

Today we’re going to transition into the secure deployment stage and talk about vulnerability assessments and penetration testing. Keep in mind that we look at web application security as an ongoing, and overlapping, process. Although we’ve divided things up into phases to facilitate our discussion, that doesn’t mean there are hard and fast lines slicing things up. For example, you’ll likely continue using dynamic analysis tools in the deployment stage, and will definitely use vulnerability assessments and penetration testing in the operations phase.

We’ve also been getting some great feedback in the comments, and will be incorporating it into the final paper (which will be posted here for free). We’ve decided this feedback is so good that we’re going to start crediting anyone who leaves comments that result in changes to the content (with permission, of course). It’s not as good as paying you, but it’s the best we can do with the current business model (for now- don’t assume we aren’t thinking about it).

As we dig into this keep in mind that we’re showing you the big picture and everything that’s available. When we close the series we’ll talk prioritization and where to focus your efforts for those of you on a limited budget- it’s not like we’re so naive as to think all of you can afford everything on the market.

Vulnerability Assessment

In a vulnerability assessment we scan a web application to identify anything an attacker could potentially use against us (some assessments also look for compliance/configuration/standards issues, but the main goal in a VA is security). We can do this with a tool, service, or combination of approaches.

A web application vulnerability assessment is very different than a general vulnerability assessment where we focus on network and hosts. In those, we scan ports, connect to services, and use other techniques to gather information revealing the patch levels, configurations, and potential exposures of our infrastructure. Since, as we’ve discussed, even “standard” web applications are essentially all custom, we need to dig a little deeper, examine application function and logic, and use more customized assessments to determine if a web application is vulnerable. With so much custom code and implementation, we have to rely less on known patch levels and configurations, and more on actually banging away on the application and testing attack pathways. As we’ve said before, custom code equals custom vulnerabilities. (For an excellent overview of web application vulnerability layers please see this post by Jeremiah Grossman. We are focusing on the top three layers- third-party web applications, and the technical and business logic flaws of custom applications).

The web application vulnerability assessment market includes both tools and services. Even if you decide to go the tool route, it’s absolutely critical that you place the tools in the hands of an experienced operator who will understand and be able to act on the results. It’s also important to run both credentialed and uncredentialed assessments. In a credentialed assessment, the tool or assessor has usernames and passwords of various levels to access the application. This allows them inside access to assess the application as if they were an authorized user attempting to exceed authority.

Tools

There are a number of commercial, free, and open source tools available for assessing web application vulnerabilities, each with varying capabilities. Some tools only focus on a few kinds of exploits, and experienced assessors use a collection of tools and manual techniques. For example, there are tools that focus exclusively on finding and testing SQL injection attacks. Enterprise-class tools are broader, and should include a wide range of tests for major web application vulnerability classes, such as SQL injection, cross site scripting, and directory traversals. The OWASP Top 10 is a good starting list of major vulnerabilities, but an enterprise class tool shouldn’t limit itself to just one list or category of vulnerabilities. An enterprise tool should also be capable of scanning multiple applications, tracking results over time, providing robust reporting (especially compliance reports), and providing reports customized local needs (e.g., add/drop scans).

Tools are typically software, but can also include dedicated appliances. Tools can run either manual scans with an operator behind them, or automatics scans for on a schedule. Since web applications change so often, it’s important to scan any modifications or new applications before deployment, as well as live applications on an ongoing basis.

Services

Not all organizations have the resources or need to buy and deploy tools to assess their own applications, and in some cases external assessments may be required for compliance.

There are three main categories of web application vulnerability assessment services:

• Fully automatic scans: These are machine-run automatic scans that don’t involve a human operator on the other side. The cost is low, but they are more prone to false positives and false negatives. They work well for ongoing assessments on a continuous basis, but due to their limitations you’ll likely still want more in-depth assessments from time to time.
• Automatic scans with manual evaluation: An automatic tool performs the bulk of the assessment, followed by human evaluation of the results and additional testing. They provide a good balance between ongoing assessments and the costs of a completely manual assessment. You get deeper coverage and more accurate results, but at a higher cost.
• Manual assessments: A trained security assessor manually evaluates your web application to identify vulnerabilities. Typically an assessor uses their own tools, then validates the results and provides custom reports. The cost is higher per assessment than the other options, but a good assessor may find more flaws.

Penetration Testing

The goal of a vulnerability assessment is to find potential avenues an attacker can exploit, while a penetration test goes a step further and validates whether attack pathways result in risk to the organization. In a web application penetration test we attempt to penetrate our own applications and determine what an attacker can do, and what the consequences might be.

Vulnerability assessments and penetration tests are highly complementary, and frequently performed together since the first step in any attack is to find vulnerabilities to exploit. The goal during the vulnerability assessment phase is to find as many of those flaws as possible, and during the penetration test to validate those flaws, determine potential damages, and prioritize remediation efforts.

That’s the key value of a penetration test- it bridges the gap between the discovered vulnerability and the exploitable asset so you can make an appropriate risk decision. We don’t just know we’re vulnerable, we learn the potential consequences of those vulnerabilities. For example, your vulnerability scan may show a SQL injection vulnerability, but when you attempt to exploit it in the penetration test it doesn’t reveal sensitive information, and can’t be used to damage the web application. On the other hand, a seemingly minor vulnerability might turn out to allow exploitation of your entire web application.

Some experts consider penetration tests important because they best replicate the techniques and goals an attacker will use to compromise an application, but we find that a structured penetration test is more valuable as a risk prioritization tool. If we think in terms of risk models, a properly performed penetration test helps fill in the potential severity/impact side of the analysis. Of course, this also means you need to focus your penetration testing program on risk assessment, rather than simply “breaking” a web application.

As with vulnerability assessments, penetration tests can include the entire vulnerability stack (from the network and operating system up through your custom application code), and both tools and services are available. Again, we’ll limit this discussion to web application specific aspects of penetration testing.

Tools

A web application penetration testing tool provides a framework for identifying and safely exploiting web vulnerabilities, and measuring or estimating their potential impact. While there are many tools used by penetration testers, most of them are point tools, rather than broad suites. An enterprise-class tool adds features to better assist the risk management process, support internal assessments, and to reduce the costs of internal penetration tests. This includes broad coverage of application vulnerability classes, “safe” techniques to exploit applications without interfering with their ongoing use, workflow, extensive reporting, and automation for ongoing assessments. One advantage of penetration testing tools is you can integrate them into the development and deployment process, rather than just assessing live web applications.

Penetration testing tools are always delivered as software and should be used by a trained operator. While parts of the process can be automated, once you dig into active exploitation and analysis of results, you need a human being involved.

When using penetration testing (or vulnerability assessment) tools, you have the choice of running them in a safe mode that reduces the likelihood of causing a service or application outage (this is true for all kinds of VA and penetration tests, not just web applications). You’ll want to use safe mode on live applications. But these tools are extremely valuable in assessing development/test environments before applications are deployed, and you shouldn’t restrict your exploit attempts to non-production systems.

Services

Unlike vulnerability assessments, we are unaware of any automated penetration testing services (although some of the automatic/manual services come close). Due to the more intrusive and fluid nature of a web application penetration test you always need a human to drive the process. Penetration testing services are typically offered as either one-off consulting projects or a subscription service for scheduled tests. Testing frequency varies greatly based on your business needs, exposure, and the nature of your web applications. Internal applications might only be assessed annually as part of a regularly scheduled enterprise-wide test.

When evaluating a penetration testing service it’s important to understand their processes, experience, and results. Some companies offer little more than a remote scan using standard tools, and fail to provide prioritized results that are usable in your risk assessment. Others simply “break into” web applications and stop as soon as they get access. You should give preference to experienced organizations with an established process that can provide sample reports (and references) that meet your needs and expectations. Most penetration tests are structured as either fixed-time or fixed-depth. In a fixed-time engagement (the most common) the penetration testers discover as much as they can in a fixed amount of time. Engagements may also divide the time used into phases- e.g. a blind attack, a credentialed attack, and so on. Fixed-depth engagements stop when a particular penetration goal is achieved, no matter how much time it takes (e.g. administrative access to the application or access to credit card numbers).

Integrating Vulnerability Assessment and Penetration Testing into Secure Deployment

Although most organizations tend to focus web application vulnerability assessments and penetration tests on externally accessible production applications, the process really begins during deployment, and shouldn’t be limited to external applications. It’s important to test applications in depth before you expose them to either the outside world or internal users. There’s also no reason, other than resources, you can’t integrate assessments into the development process itself at major milestones.

Before deploying an application, set up your test environment so that it accurately reflects production. Everything from system configurations and patch levels, up through application connections to outside services must match production or results won’t be reliable. Then perform your vulnerability assessment and penetration test. If you use tools, you’ll do this with your own (appropriately trained) personnel. If you use a service, you’ll need to grant them remote access to your test environment. Since few organizations can afford to test every single application to the same degree, you’ll want to prioritize based on the exposure of the application, its criticality for business operations, and the sensitivity of the data it accesses.

For any major externally accessible application you’ll want to engage a third party for both VA and penetration testing before deployment. If your business relies on it for critical operations, and it’s publicly accessible, you really want to get an external assessment.

Once an application is deployed, your goal should be to perform at least a basic assessment on any major modifications before they commit. For critical applications, engage a third-party service for ongoing vulnerability assessments, and periodic penetration tests (at least annual or after major updates) on top of your own testing.

We know not all of you have the resources to support internal and external tools and services for VA and penetration testing on an ongoing basis, so you’ll need to adapt these recommendations for your own organizations. We’ve provided a high level overview of what’s possible, and some suggestions on where and how to prioritize.

In our next post we’ll close out our discussion of web application security technologies by looking at web application firewalls and monitoring tools. We’ll then close out the series by showing you how to put these pieces together into a complete program, how to prioritize what you really need, and how to fit it to the wild world of web application development.

–Rich

WAF vs. Secure Code vs. Dead Fish

By Rich

I've been slowly catching up on my reading after months of near-nonstop travel, and this post over at Imperviews caught my eye. Ignoring the product promotion angle, it raises one of my major pet peeves these days. I'm really tired of the Web Application Firewall vs. secure coding debate, never mind using PCI 6.6 to justify one over the other for security effectiveness. It's like two drunk cajuns arguing over the relative value of shrimp or pork in gumbo- you need both, and if either is spoiled the entire thing tastes like sh&t. You also can't dress up the family dog and fish in a pinch, use them as substitutes, and expect your kids to appreciate either the results or use of resources (resulting gumbo or the loss of Rover).

Here's the real deal-

Secure coding is awesome and you need to adopt a formal process if you produce any meaningful volume of code. But it takes a ton of resources to get to the old code (which you should still try to do), and can't account for new vulnerability classes. Also, people screw up... even when there are multiple layers to detect or prevent them from screwing up.

On the other hand, WAFs need to get a hell of a lot better. We're seeing some positive advancements, as I've written about before, but they still can't stop all vulnerabilities, can't stop logic flaws and certain other categories of attack, can't deal with the browser end, and I hear a lot of complaints about tuning (while I think liking WAFs with Vulnerability Assessment is a great start on this problem, we're just at the start of that race).

I absolutely hate to tell you to buy more than you need, but if you have a major web presence you likely need both these days, in the right combination (plus a few other things).

If you don't have the resources for both, I suggest two options. First, if you are really on the low end of resources, use hosted applications and standard platforms as much as possible to limit your custom coding. Then, make sure you have kick ass backups. Finally, absolutely minimize the kinds of information and transaction you expose to the risk of web attacks- drop those ad banners, minimize collecting private information, and validate transactions on the back end as much as possible.

If you do have some more resources available, I suggest starting with a vulnerability assessment (not a cheap ass bare-bones PCI scan, but something deeper), and using that to figure out where to go next.

Yes- we are eating our own dog food on this one. The blog is hosted using a standard platform. We know it's vulnerable, so we've minimized the attack surface as best we can and make sure we have backups of all the content. I've been pleasantly surprised we haven't been nailed yet, but I expect it to happen eventually. None of our sensitive operations are on that server, and we've pulled email and our other important stuff in house.

Early next year we're going to be launching some new things, and we will again go with remote hosting (on a more powerful platform). This time, we are switching to a more secure platform than Wordpress (Expression Engine) and will pay for a full vulnerability assessment and penetration test (at least annually, or when any major new components come online). We may perform some financial transactions, and we'll use an external provider for that. A WAF is out of budget for us, so we'll focus on minimizing our exposure and manually fixing problems discovered by ongoing assessments. We also plan on using as little custom code as possible.

But seriously- I'm tired of this debate. Both options have value, they aren't exclusionary, and which you need depends on what you are doing and how many resources you have.

Eventually we'll get a better lock on this problem, but that's a few years out.

–Rich

How To Tell If Your PCI Scanning Vendor Is Dangerous

By Rich

I got an interesting email right before I ran off on vacation from Mark on a PCI issue he blogged about:

13. Arrangements must be made to configure the intrusion detection system/intrusion prevention system (IDS/IPS) to accept the originating IP address of the ASV. If this is not possible, the scan should be originated in a location that prevents IDS/IPS interference. snip... I understand what the intention of this requirement is. If your IPS is blacklisting the scanner IP's then ASVs don't get a full assessment because they are a loud and proud scan rather than a targeted attack... However, blindly accepting the originating IP of the scanner leaves the hosts vulnerable to various attacks. Attackers can simply reference various public websites to see what IP addresses they need to use to bypass those detective or preventive controls.

I figured no assessor would ask their client to open up big holes just to do a scan, but lo and behold, after a little bit of research it turns out this is surprisingly common. Back to email:

It came up when I was told by my ASV "Authorized scanning vendor" that I had to do exclude their IPs. They also provided me with the list of IP's to exclude. Both [redacted] and [redacted] have told me I needed to do bypass the IDS. When I asked about the exposure they were creating, both told me that their "other customers" do this and it isn't a problem for them.

If your ASV can't perform a scan/test without having you turn off your IDS/IPS, it might be time to look for a new one. Especially if their source IPs are easy to figure out.

For the record, "everyone else does it" is the dumbest freaking reason in the book. Remember the whole jumping off a bridge thing your mom taught you?

–Rich