Login  |  Register  |  Contact

App Security

Wednesday, April 01, 2009

New Application Security Certification Launched

By Rich

We’ve been talking a lot about application security since we started this blog, and one thing we’ve been tracking closely are training and certification programs. 200903312137.jpg While we couldn’t talk about it, we’ve been quietly involved with the Institute for Certified Application Security Specialists. We reviewed the program during development, and were overall pretty impressed. It has very similar requirements to the CSSLP, but is more cost effective for security practitioners… something we can all appreciate in this economy. Believe it or not, despite my not-infrequent diatribes against various certifications, I actually went through the process myself and am fully certified. What I really appreciate is how pragmatic the program is, and how it really reflects the operational realities of application security.

You can get more information at the Institute for Certified Application Security Specialists, and as a member of the affiliate program Securosis readers receive a 10% discount.

Oh- and don’t forget to join the LinkedIn Group!


Tuesday, March 31, 2009

Comments on “Containing Conficker”

By Adrian Lane

As you have probably read, a method for remotely detecting systems infected with the Conficker worm was discovered by Felix Leder and Tillmann Werner. They have been working with Dan Kaminisky, amongst others, to come up with a tool to detect the worm and give IT organizations the ability to protect themselves. This is excellent news. The bad news is how unprepared most applications are to handle threats like this. Earlier this morning, the guys at The Honeynet Project were kind enough to forward Rich and myself a copy of their Know Your Enemy: Containing Conficker paper. This is a very thorough analysis of how the worm operates. I want keep my comments on this short, and simply recommendation strongly that you read the paper. If you are in software development, you need to read this paper.

Their analysis of Conficker illustrates that the people who wrote it are far ahead of your typical application development team in their understanding of application security. Developers need to understand the approach that attackers are taking, understand the dedication to their craft these guys are exhibiting, and increase their own knowledge and dedication if they are going to have a chance of producing code that can counter these types of threats.

Is Conficker a well-written piece of code? Is it architected well? No idea. But it is clear that each iteration has advanced their three core functions (find & infect, maintain, & defend) and had this flexibility in mind from the begining. Look at how Conficker uses identification techniques to protect itself in avoid downloading the wrong/malicious patches to their worm. And check out the examination of incoming requests to help protect their now infected system from other viruses. This should serve as an example of how to write internal monitoring code to detect exploit attempts (see section 4), either in lieu of a full blown patch, or as self-defending code at critical points, or both. And it is done in a manner that gives them a generic tool that, when updated, will be an effective anti-malware tool. Neat, huh? The authors have a pretty good understanding of randomness and used multiple sources, not only to get better randomness, but to avoid an attack on any one- smart. These are really good application security practices that very few software authors actually put into practice. Heck, most web applications trust everything that comes in, and it looks like the authors of Conficker understand that you must trust nothing!

Once again, if you are a software developer or IT practitioner, read the paper. The research that Felix and Tillmann have put into this is impressive. They have proof points for everything they believe to be true about the worm’s behavior, and have stuck with the facts. This is really time consuming, difficult work. Excellent job, guys!

–Adrian Lane

Tuesday, March 24, 2009

Security Speed-bumps

By Adrian Lane

Reading yet another comment on yet another blog about “what good is ABC technology because I can subvert the process” or “we should not use XYZ technology because it does not stop the threats” … I feel a rant coming on. I get seriously annoyed when I hear these blanket statements about how some technologies are no good because they can be subverted. I appreciate zeal in researchers, but am shocked by people’s myopia in applied settings. Seriously, is there any technology that cannot be compromised?

I got a chance to chat with an old friend on Friday and he reminded me of a basic security tenet … most security precautions are nothing more than ‘speed bumps’. They are not fool-proof, not absolute in the security that they offer, and do not stand unto themselves without support. What they do is slow attackers down, make it more difficult and expensive in time, money, and processing power to achieve their goals. While I may not be able to brute force and already encrypted file, I can subvert most encryption systems, especially if I can gain access to the host. Can I get by your firewall? Yes. Can I get spam through your email filter? Absolutely. Can I find holes in your WAF policy set? Yep. Write malware that goes undetected, escalate user privileges, confuse your NAC, poison your logs, evade IDS, compromise your browser? Yep. But I cannot do all of these things at the same time. Some will slow me down while others detect what I am doing. With enough time and attention there are very few security products or solutions that would not succumb to attack under the right set of circumstances, but not all of them at one time. We buy anti-spam, even if it is not 100% effective, because it makes the problem set much smaller. We try not to click email links and visit suspect web sites because we know our browsing sessions are completely at risk. When we have solid host security to support encryption systems, we drop the odds of system compromise dramatically.

If you have ever heard me speak on security topics, you will have heard a line that I throw into almost every presentation: embrace insecurity! If you go about selecting security technologies thinking that they will protect you from all threats under all circumstances, you have already failed. Know that all your security measures are insecure to some degree. Admit it. Accept it. Understand it. Then account for it. One of the primary points Rich and I were trying to make in our Web Application Security paper was that there are several ways to address most issues. And it’s like fitting pieces of a puzzle together to get reasonable security against your risks in a cost effective manner. What technologies and process changes you select depend upon the threats you need to address, so adapt your plans such that you cover for these weaknesses.

–Adrian Lane

Saturday, March 07, 2009

Director of National Cyber-Security Center Resigns

By Adrian Lane

A couple days ago I posted some thoughts on Data Security and the US Government, how I perceive the role of Cybersecurity, and what I suspected would be a difficult challenge as the Cybersecurity team was set up at cross-purposes with the intelligence community. Today the Wall Street Journal released an article on the resignation of National Cybersecurity Chief Rod Beckstrom. In a case of “even a blind squirrel occasionally finds a nut”, my estimate of internal conflict appears to already be going on. In his resignation letter, Mr. Beckstrom stated that the “NSA currently dominates most national cyber efforts” and “The intelligence culture is very different than a network operations or security culture”. The WSJ focuses on privacy and separation of power issues with additional comments from Mr. Beckstrom: “the threats to our democratic process … if all top level network security and monitoring are handled by any one organization”.

The resignation letter has a different feel and focus, pointing out that there was a general lack of support for the NCSC, and the specific ways Beckstrom feels his organizations was subjugated. If you have interest in this subject, you will want to read his resignation letter, as it contains more information. It also lists a couple methods by which the NSA can subtly (sneakily?) affect the effectiveness of Cybersecurity efforts that I did not mention in my post. Quite frankly I am surprised that the National Cybersecurity Center could somehow manage to only get 5 fully funded days of operation, but if true, this demonstrates the challenges faced by NCSC.

This could get ugly unless both sides understand that each organization can benefit the other, and realize the goals and agendas do not necessarily need to be at the expense of each other. Concessions have to be made, otherwise this is an expensive and ugly turf war and the entire security problem- which is quickly becoming a US government security problem- continues to fester.

–Adrian Lane

More on PDF /JBIGS2Decode Issue

By Adrian Lane

Via Slashdot, I just ran across Didier Stevens post on how to automate the JBIG2decode vulnerability in PDF documents. There is a video on the site where he runs through three scenarios to exercise the vulnerability - Manually starting up Reader, viewing a thumbnail PDF, and then automatic execution by simply visiting the page with the malicious document through Windows Explorer Shell Extensions, and shows you the results in the debugger. It’s worth the view.

When you install Adobe Acrobat Reader, a Column Handler Shell Extension is installed. A column handler is a special program (a COM object) that will provide Windows Explorer with additional data to display (in extra columns) for the file types the column handler supports. The PDF column handler adds a few extra columns, like the Title. When a PDF document is listed in a Windows Explorer windows, the PDF column handler shell extension will be called by Windows Explorer when it needs the additional column info. The PDF column handler will read the PDF document to extract the necessary info, like the Title, Author.

I also ran across another technical analysis here. As you don’t need to do anything other that drop onto an infected site, this is a pretty serious issue. There is supposed to be a patch available later this month. The more I look at this, the more I think it may be a good idea to disable Reader until there is a patch. There are some instructions on how to do this on the PC Mag site, and some additional information you might find helpful as well.

–Adrian Lane

Thursday, March 05, 2009

My Perspective on Data Security and the US Government

By Adrian Lane

During the recent podcast I did with Rich, I made a couple throw-away comments about the selection of Melissa Hathaway as cybersecurity advisor. A lot of ideas went into those comments and a few articles that I have read that brought to the fore several issues I have had ideas rolling around in my head for the last couple of years. In fact I have written this post a couple of times over the last year and deleted it because I thought it would be perceived as too political. My goal is not political commentary rather trying to provide perspective about the evolution of data security, but sometimes the two are linked so tightly together it is difficult to fully separate.

The subject I want to discuss is the general state of basic underlying security of our electronic infrastructure, and the role that the government plays in Cyber Security. What got me going on this subject yet again was several articles that I ran across in the last month. The first was an article referenced by Bruce Schneier’s blog on The Register that talks about the NSA’s attempt to eavesdrop on Skype, which I am not sure they confirmed but highly believable. The second is the appointment of Melissa Hathaway, and while it is only being hinted at in this piece by USA Today, her comments indicate efforts at odds with other US intelligence organizations. The final article that urged me to rewrite this post was the following piece on Wired’s Threat Level Blog that the NSA wants to oversee cybersecurity.

There is a strange push and pull going on here, because part of our government wants our entire electronic infrastructure to be both secure and private. They recognize the the Internet is a huge global marketplace for science and commerce, and is often leveraged by public entities as well, therefore it is in our best interest to have it secured to protect citizens & organizations alike. This is echoed in Hathaway’s comments. Conceptually this would reduce fraud globally, which costs companies billions of dollars every year. On the other side of the coin, strong electronic security makes intelligence gathering through eavesdropping difficult to impossible, and often requires secondary assistance to gather (insider cooperation, back doors in code and devices, cracking, etc) the same information, only at a higher cost.

So what’s the problem? Good security on communications and infrastructure worldwide makes intelligence gather much, much harder. The people I have spoken with who have worked for or with US & British Intelligence organizations all share the same view that a secure communications infrastructure is facing stiff challenges from within our own government. A few years ago, Mr. Stephen Squires, who is/was at the time Chief Scientist for HP, spoke at Stanford Luncheon I attended about the evolution of computer security in the US. In a nutshell, he felt that cryptography would have solved most of the issues of privacy and security we have today, and today’s vendors with their point solutions were less than Band-Aids on gunshot wounds. Encryption could have easily been built into routers, phone switches, Ethernet cards and the like to ensure safe data transmission. Encryption could have been built into business applications to offer considerably higher security for data in motions and data at rest. He went on to say this was “discouraged” in various direct and indirect ways by our own government. He cited many examples of influence; the way bids are done, project specifications, funding, public-private partnerships, and most notably, US export controls on cryptography. A decade ago this was a common topic most of the crypto guys I have had the pleasure of meeting, and they were mostly frustrated by the US’s unwillingness to have all data and communications encrypted and secured.

For those of you not familiar with what I am talking about, in the mid-90s, you could obtain cryptographic algorithms in papers and textbooks, but if you shipped encryption technology, you were going to be brought up on charges for illegal disbursement of munitions. When I got my first real job involving security, I had to be careful that I did not accidentally include a foreign national on the “CC:” line of an email that included the Blowfish variant we were working on as I could have been arrested. It’s code, for &!^@ sake! But the US Government was quite serious about this and has hindered the deployment of some security technologies on national infrastructure as well as allowing exportation, and have done little to lead in an area where they are in a perfect position to set an example for private industry. I am not sure what degree the majority of security professionals out there understand how rabid our government intelligence agencies are about encryption, but many historians view US victory in WW2 was due in large part to breaking the Japanese encryption codes, as well as the British efforts to reconstruct the Enigma cipher. While these are nice historical references, few are aware of more recent cases with the British in Falklands war and governments in the middle east were breached by the ability to break or bypass cryptography. This lesson is not lost on the Intelligence community, and who I would expect nothing less than to look for any advantage they can get.

What is good for business security is considered bad for our spies. The concept from the governmental perspective was the benefits that the US Intelligence Services derive from lax security and encryption was a huge competitive advantage, so impediments into quality encryption technologies being widely available was to be discouraged. And when I have this discussion with people, they are usually thinking about Echelon, or the PROMIS variants that perform data analysis, but just two angles of using intelligence. We read all the time about Chinese or Russians breaking into US Government computers; is there some compelling reason to think this is not going on in the other direction? Another way to think about this: You’re the IT manager for the government of Derka Derkastan, and you install a bunch of Kludgy Corp software and “proprietary” cryptographic systems. You have in essence done a favor to every intelligence organization who wants to know what you are up to. Security may be good enough to keep “Script-Kiddies” at bay, but not professionals. And today, all commerce on the Internet is under attack from professionals.

As a science, we know how to develop encryption technologies that work. And encryption is very well suited for solving several security and privacy issues in communication, authentication, data storage and so on. In many cases where we see data breaches, the use of the technology has either been misapplied (trying to solve the wrong problem), sloppily applied (bad execution), or inconsistently applied (some parts of the infrastructure, not others). Don’t get me wrong, I am not saying Cryptography is a panacea that solves all security ills. It is just one link in the chain and needs the support of good access controls, assessment, key management and auditing technologies as well, but good crypto solves a lot of critical issues!

So do I think data security is still hampered by part of the US Government? Yes, I think there is evidence to support this. But now that the US Government is a participant in the banking and global finance industries through recent (ahem) investments, it will be interesting to see if they put serious efforts into data security as fraud will be costing them billions of dollars. When I see the Cyber Security advisor make statements that they are going to “fix critical infrastructure networks against on-line threats”, followed by the NSA “wanting to control Cyber-Security”, I know the political jockeying has only just begun. I worry because the NSA’s charter is not the promotion good data security. Hathway’s challenge is not finding technology and methodologies to secure the infrastructure, because we have most of that today. It is an issue of wide-spread adoption, and her challenge will be getting that adoption in the face of conflicting agendas. We are at a time when we need to “raise the bar” when it comes to security of our infrastructure, meaning systems that are beyond trivial to break, and that means a more strategic approach than vendors and corporations have been willing to take thus far.

–Adrian Lane

Thursday, February 19, 2009

New Database Configuration Assessment Options

By Adrian Lane

Oracle has acquired mValent, the configuration management vendor. mValent provides an assessment tool to examine the configuration of applications. Actually, they do quite a bit more than that, but I wanted to focus on the value to database security and compliance in this post. This is a really good move on Oracle’s part as it fills a glaring hole that they have had for some time in their security and compliance offerings. I have never understood why Oracle did not provide this as part of OEM as every Oracle event I have been to in the last 5 years has sessions where DBA’s are swapping scripts to assess their database. Regardless, they have finally filled the gap. It provides them with a platform to implement their own best practice guidelines, and gives customers a way to implement their own security, compliance and operational policies around the database and (I assume) other application platforms. Sadly, many companies have not automated their database configuration assessments, and the market remains wide open, and this is a timely acquisition.

While the value proposition for this technology will be spun by Oracle’s marketing team in a few dozen different ways (change management, compliance audits, regulatory compliance, application controls, application audits, compliance automation, etc), don’t get confused by all of the terms. When it comes down to it, this is an assessment of application configuration. And it does provide value in a number of ways: security, compliance and operations management. The basic platform can be used in many different ways all depending upon how you bundle the policy sets and distribute reports.

Also keep in mind that a ‘database audit’ and ‘database auditing’ are two completely different things. Database auditing is about examining transactions. What we are talking about here is how the database is configured and deployed. To avoid the deliberate market confusion on the vendors part, here at Securosis we will stick to the terms Vulnerability Assessment and Configuration Assessment to describe the work that is being performed.

Tenable Network Security has also announced on their blog that they now have the ability to perform credentialed scans of the database. This means that Nessus is no longer just a pen-test style patch level checker, but a credentialed/peer based configuration assessment. By ‘Credentialed’ I mean that the scanning tool has a user name and password with some access rights the database. This type of assessment provides a lot more functionality because there is a lot more information available to you that is not available through a penetration test. This is necessary progression for the product as the ports, quite specifically the database ports, no longer return sufficient information for a good assessment of patch levels, or any of the important information for configuration.

If you want to produce meaningful compliance reports, this is the type of scan you need to provide. While I occasionally rip Tenable Security as this is something they should have done two years ago, it is really a great advancement for them as it opens up the compliance and operation management buying centers. Tenable must be considered a serious player in this space as this is a low cost, high value option. They will continue to win market share as they flesh out the policy set to include many of the industry best practices and compliance tests.

Oracle will represent an attractive option for many customers, and they should be able to immediately leverage their existing relations. While not cutting edge or best-of -breed in this class, I expect many customers will adopt as it will be bundled with what they are already buying, or the investment is considered lower risk as you are going with the worlds largest business software vendors. On the opposite end of the spectrum, companies who do not view this as business critical but still want thorough scans will employe the cost effective Tenable solution. Vendors like Fortinet, with their database security appliance, and Application Security’s AppDetective product, will be further pressed to differentiate their offerings to compete with the perceived top end and bottom ends of the market. Things should get interesting in the months to come.

–Adrian Lane

Friday, January 30, 2009

Policies and Security Products

By Adrian Lane

Where do the policies in your security product come from? With the myriad of tools and security products on the market, where do the pre-built policies come from? I am not speaking of AV in this post- rather looking at IDS, VA, DAM, DLP, WAF, pen testing, SIEM, and many others that use a set of policies to address security and compliance problems. The question is who decides what is appropriate? On every sales engagement, customer and analyst meeting I have ever participated in for security products, this was a question.

This post is intended more for IT professional who are considering security products, so I am gearing for that audience. When drafting the web application security program series last month, a key topic that kept coming up over and over again from security practitioners was: “How can you recommend XYZ security solution when you know that the customer is going to have to invest a lot for the product, but also a significant amount in developing their own policy set?” This is both an accurate observation and the right question to be asking. While we stand by our recommendations for reasons stated in the original series, it would be a disservice to our IT readers if we did not discuss this in greater detail. The answer is an important consideration for anyone selecting a security tool or suite.

When I used to develop database security products, policy development was one of the tougher issues for us to address on the vendor side. Once aware of a threat, on average it took 2.5 ‘man-days’ to develop a policy with a test case and complete remediation information [prior to QA]. This becomes expensive when you have hundreds of policies being developed for different problem sets. It was a common competitive topic to discuss policy coverage and how policies were generated, and a basic function of the product, so most every vendor will invest heavily in this area. More, most vendors market their security ‘research teams’ that find exploits, develop test code, and provide remediation steps. This domain expertise is one of the areas where vendors provide value in the products that they deliver, but when it comes down to it, vendor insight is fraction of the overall source of information. With monitoring and auditing, policy development was even harder: The business use cases were more diverse and the threats not completely understood. Sure we could return the ubiquitous who-what-when-where-to-from kind of stuff, but how did that translate to business need?

If you are evaluating products or interested in augmenting your policy set, where do you start? With vulnerability research, there are several resources that I like to use:

Vendor best practices - Almost every platform vendor, from Apache to SAP, offer security best practices documents. These guidelines on how to configure and operate their product form the basis for many programs. These cover operational issues that reduce risk, discuss common exploits, and reference specific security patches. These documents are updated during each major release cycle, so make sure you periodically review for new additions, or how they recommend new features be configured and deployed. What’s more, while the vendor may not be forthcoming with exploit details, they are the best source of information for remediation and patch data.

CERT/Mitre - Both have fairly comprehensive lists of vulnerabilities to specific products. Both provide a neutral description of what the threat is. Neither had great detailed information of the actual exploit, not will they have complete remediation information. It is up to the development team to figure out the details.

Customer feedback/peer review - If you are a vendor of security products, customer have applied the policies and know what works for them. They may have modified the code that you use to remediate a situation, and that may be a better solution than what your team implemented, and/or it may be too specific to their environment for use in a generalized product. If you are running your own IT department, what have your peers done? Next time you are at a conference or user group, ask. Regardless, vendors learn from other customers what works for them to address issues, and you can too.

3rd party relationships (consultants, academia, auditors) - When it comes to development of policies related to GLBA or SOX, which are outside the expertise of most security vendors, it’s particularly valuable to leverage third party consultative relations to augment policies with their deep understanding of how best to approach the problem. In the past I have used relationships with major consulting firms to help analyze the policies and reports we provided. This was helpful, as they really did tell us when some of our policies were flat out bull$(#!, what would work, and how things could work better. If you have these relationships already in place, carve out a few hours so they can help review and analyze policies.

Research & Experience - Most companies have dedicated research teams, and this is something you should look for. They do this every day and they get really good at it. If your vendor has a recognized expert in the field on staff, that’s great too. That person may be quite helpful to the overall research and discovery process of threats and problems with the platforms and products you are protecting. The reality is that they are more likely on the road speaking to customers, press and analysts rather than really doing the research. It is good that your vendor has a dedicated team, but their experience is just one part of the big picture.

User groups - With many of the platforms, especially Oracle, I learned a lot from regional DBAs who supported databases within specific companies or specific verticals. In many cases they did not have or use a third party product, rather they had a bunch of scripts that they had built up over many years, modified, and shared with others. They shared tips on not only what they were required to do, but how they implemented them. This typically included the trial-and-error discussion of how a certain script or policy was evolved over time to meet timeliness or completeness of information requirements from other team members. Use these groups and attend regional meetings to get a better idea of how peers solve problems. Amazing wealth of knowledge, freely shared.

General frameworks - To meet compliance efforts, frameworks commonly provide checklists for compliance and security. The bad news is that the lists are generic, but the good news is they provides a good start for understanding what you need to consider, and help prepare for pre-vendor engagements and POCs.

Compliance - Polices are typically created to manage compliance with existing policies or regulations. Compliance requirements allow some latitude for how you interpret how a PCI or FISMA applies to your organization. What works, how it is implemented, what the auditors find suitable, and what is easy for them to use all play a part in the push & pull of policy development, and one of the primary reasons to consider this effort as added expense to deploying third party products.

I want to stress that you should use this as a guide to review the methods that product vendors use to develop their policies, but my intention is to make sure you clearly understand that you will need to develop your own as well. In the case of web application security, it’s you application, and it will be tough to avoid. This post may help you dig through vendor sales and marketing literature to determine what can really help to you and what is “pure puffery”, but ultimately you need to consider the costs of developing your own policies for the products you choose. Why? You can almost never find off-the-shelf polices that meet all of your needs. Security or compliance may not be part of your core business, and you may not be a domain expert in all facets of security, but for certain key areas I recommend that you invest in supplementing the off-the-shelf policies included with your security tools. Policies are best if they are yours, grounded in your experience, and tuned to your organizational needs. They provide historical memory, and form a knowledge repository for other company members to learn from. Policies can guide management efforts, assurance efforts, and compliance efforts. Yes, this is work, and potentially a lot of work paid in increments over time. If you do not develop your own policies, and this type of effort is not considered within your core business, then you are reliant on third parties (service providers or product vendors) for the production of your policies.

Hopefully you will find this helpful.

–Adrian Lane

Submit A Top Ten Web Hacking Technique

By Rich

Last week Jeremiah Grossman asked if I’d be willing to be a judge to help select the Top Ten Web Hacking Techniques for 2008. Along with Chris Hoff (not sure who that is), H D Moore, and Jeff Forristal.

Willing? Heck, I’m totally, humbly, honored.

This year’s winner will receive a free pass to Black Hat 2009, which isn’t to shabby.

We are up to nearly 70 submissions, so keep ‘em coming.


Saturday, January 24, 2009

How Much Security Will You Tolerate?

By Adrian Lane

I have found a unique way to keep anyone from using my iMac. While family & friends love the display, they do not use my machine. Many are awed that they can run Windows in parallel to the Mac OS, and the sleek appearance and minimal footprint has created many believers- but after a few seconds they step away from the keyboard. Why? Because they cannot browse the Internet. My copy of Firefox has NoScript, Flashblock, cookie acknowledgement, and a couple of other security related ad-ons. But having to click the Flash logo, or to acknowledge a cookie, is enough to make them leave the room. “I was going to read email, but I think I will wait until I fly home”.

I have been doing this so long I never even notice. I never stopped to think that every web page requires a couple extra mouse clicks to use, but I always accepted that it was worth it. The advantages to me in terms of security are clear. And I always get that warm glow when I find myself on a site for the first time and see 25 Flash icons littering the screen and a dozen cookie requests for places I have never heard of. But I recognize that I am in the minority. The added work seems to so totally ruin the experience and completely turn them off to the Internet. My wife even refused to use my machine, and while I think the authors of NoScript deserve special election into the Web Security Hall of Fame (Which given the lack of funding, currently resides in Rich’s server closet), the common user thinks of NoScript as a curse.

And for the first time I think I fully understand their perspective, which is the motivation for this post. I too have discovered my tolerance limit. I was reading rsnake’s post on RequestPolicy Firefox extension. This looks like a really great idea, but acts like a major work inhibitor. For those not fully aware, I will simply say most web sites make requests for content from more than just one site. In a nutshell you implicitly trust more than just the web site you are currently visiting, but whomever provides content on the page. The plugin’s approach is a good one, but it pushed me over the limit of what I am willing to accept.

For every page I display I am examining cookies, Flash, and site requests. I know that web security is one of the major issues we face, but the per-page analysis is not greater than the time I spend on many pages looking for specific content. Given that I do a large percentage of research on the web, visiting 50-100 sites a day, this is over the top for me. If you are doing any form of risky browsing, I recommend you use it selectively. Hopefully we will see a streamlined version as it is a really good idea.

I guess the question in my mind is how much security will we tolerate? Even security professionals are subject to the convenience factor.

–Adrian Lane

Tuesday, January 06, 2009

Building a Web Application Security Program, Part 8: Putting It All Together

By Adrian Lane

‘Whew! This is our final post in this series on Building a Web Application Security Program (Part 1, Part 2, Part 3, Part 4, Part 5, Part 6, Part 7), and it’s time to put all the pieces together. Here are our guidelines for designing a program that meets the needs of your particular organization. Web application security is not a “one size fits all” problem. The risks, size, and complexity of the applications differ, the level of security awareness among team members varies, and most importantly the goals of each organization are different.

In order to offer practical advice, we needed to approach program development in terms of typical goals. We picked three use cases to represent common challenges organizations face with web app security, and will address those use cases with appropriate program models. We discuss a mid-sized firm tackling a compliance mandate for the first time, a large enterprise looking to improve security across customer-facing applications, and a mid-to-large organization dealing with security for internal applications. Each perspective has its own drivers and assumptions, and in each scenario different security measures are already in place, so the direction of each program will be different. Since we’ve been posting this over a series of weeks, before you dig in to this post we recommend you review Part 4: The Web Application Security Lifecycle which talks about all tools in all phases. First we describe the environment for each case, then overall strategy and specific recommendations.

Large Enterprise with Customer Facing Web Applications

For our first scenario, let’s consider a large enterprise with multiple customer-facing web applications. These applications evolved to offer core business functions and are a principal contact point with customers, employees, and business partners. Primary business drivers for security are fraud reduction, regulatory compliance, and service reliability as tangible incentives. Secondary factors are breach preparedness, reputation preservation, and asset protection secondary – all considerations for security spending. The question is not whether these applications need to be secured, but how. Most enterprises have a body of code with questionable security, and let’s be totally honest here- these issues are flaws in your code. No single off-the-shelf product is going to magically make your application secure, so you invest not only in third-party security products, but also in improvements to your own development process which improve the product with each new release.

We assume our fictitious enterprise has an existing security program and the development team has some degree of maturity in their understanding of security issues, but how best to address problems is up for debate. The company will already have a ‘security guy’ in place, and while security is this guy’s or gal’s job, the development organization is not tasked with security assessments and problem identification. Your typical CISO comes from a network security background, lacks a secure code development background, and is not part of this effort. We find their security program includes vulnerability assessment tools, and they have conducted a review of the code for typical SQL injection and buffer overflow attacks. Overall, security is a combination of a couple third-party products and the security guy pointing out security flaws which are patched in upcoming release cycles.

Recommendations: The strategy is to include security within the basic development process, shifting the investment from external products to internal products and employee training. Tools are selected and purchased to address particular deficiencies in team skill or organizational processes. Some external products are retained to shield applications during patching efforts.

Training, Education, and Process Improvements: The area where we expect to see the most improvement is the skill and awareness of the web application development team. OWASP’s top flaws and other sources point out issues that can be addressed by proper coding and testing … provided the team knows what to look for. Training helps staff find errors and problems during code review, and iteratively reduces flaws through the development cycle. The development staff can focus on software security and not rely on one or two individuals for security analysis.

Secure SDLC: Knowing what to do is one thing, but actually doing it is something else. There must be an incentive or requirement for development to code security into the product, assurance to test for compliance, and product management to set the standards and requirements. Otherwise security issues get pushed to the side while features and functions are implemented. Security needs to be part of the product specification, and each phase of the development process should provide verification that the specification is being met through assurance testing. This means building security testing into the development process and QA test scenarios, as well as re-testing released code. Trained development staff can provide code analysis and develop test scripts for verification, but additional tools to automate and support these efforts are necessary, as we will discuss below.

Heritage Applications: Have a plan to address legacy code. One of the more daunting aspects for the enterprise is how to address existing code, which is likely to have security problems. There are several possible approaches for addressing this, but the basic steps are 1) identification of problems in the code, 2) prioritization on what to fix, and 3) planning how to fix individual issues. Common methods of addressing vulnerabilities include 1) rewriting segments of code, 2) method encapsulation, 3) temporary shielding by WAF (“secure & patch”), 4) moving SQL processing & validation into databases, 5) discontinuing use of insecure features, and 6) introduction of validation code within the execution path. We recommend static source code analysis or dynamic program analysis tools for the initial identification step. These tools are cost-effective and suitable for scanning large bodies of code to locate common risks and programming errors. They detect and prioritize issues, and reduce human error associated with tedious manual scanning by internal or external parties. Analysis tools also help educate staff about issues with certain languages and common programming patterns. The resulting arguments over what to do with 16k insecure occurrences of IFRAME are never fun, but acceptance of the problem is necessary before it can be effectively addressed.

External Validation: Periodic external review, through vulnerability assessment, penetration testing or source code review, is highly recommended . Skilled unbiased professionals with experience in threat analysis often catch items items which slip by internal scans, and can help educate development staff on different threat vectors. Plan on external penetration testing on a quarterly or biannual basis- their specific expertise and training goes far beyond the basics threats, and trained humans monitoring the output of sophisticated tools are very useful for detecting weaknesses that a hacker could exploit. We recommend the use of static testing tools for internal testing of code during the QA sweep, with internal penetration testing just prior to deployment so they can fully stress the application without fear of corrupting the production environment. Major releases should also undergo an external penetration test and review before deployment.

Blocking: This is one area that will really depend upon the specifics of your organization. In the enterprise use case, plan on using a Web Application Firewall. They provide basic protection and give staff a chance to remove security issues from the application. You may find that your code base is small and stable enough that you do not need WAF for protection, but for larger organizations this is not an option. Development and patching cycles are too long and cumbersome to counter threats in a reasonable timeframe. We recommend WAF + VA because in combination, they can relieve your organization from much of the threat research and policy development work for firewall rules. If your staff has the skill and time to develop WAF policies specific to your organization, you get customized policies at slightly greater expense in development costs. WAF isn’t cheap, so we don’t take this recommendation lightly, but it provides a great deal of flexibility in how and when threats are dealt with, today and as new threats evolve.

We recommend you take steps to improve security in every part of the development process. We are focused on improvements to the initial phases of development, as the impact of effort is greatest here, but we also recommend at the very least external assistance, and if budget allows, blocking. These later recommendations fill in other areas that need coverage, with penetration testing and web application firewalls. The risks to the enterprise are greater, the issues to overcome are more complex, and the corresponding security investment will therefore be larger. This workflow process should be formally documented for each stage of an application’s lifecycle- from development through ongoing maintenance- with checkpoints for major milestones. Security shouldn’t, and can’t, be responsible for each stage, but should carry responsibility for managing the program and making sure the proper process is followed and maintained.

Mid-sized firm and PCI Compliance


If we are discussing web application security and compliance, odds are we are talking about the Payment Card Industry’s Data Security Standard (PCI-DSS). No other compliance standard specifies steps to secure web applications like the PCI standard does. We can grouse about ambiguities and ways that it could be improved, but PCI is clearly the most widespread driver for web application security today, which is why our second use case is a mid-sized firm that needs to secure its web applications to satisfy PCI-DSS.

The profile for our company is a firm that generates a large portion of their revenue through Internet sales, and recent growth has made them a Tier 3 merchant. The commerce web site is relatively new (< 3 years) and the development team is small and not trained in security. Understanding the nuances of how criminals approach breaking code is not part of the team’s skill set. PCI compliance is the mandate, and the team knows that they are both missing the basic requirements and susceptible to some kinds of attacks. The good news is that the body of code is small, and the web application accounts for a significant portion of the company’s revenue, so management is supporting the effort.

In a nutshell, the PCI Data Security Standard is a security program specifically for companies that process credit card transactions for Internet commerce. In terms of compliance regulations, PCI-DSS requirements are clearer than most, making specific requirements for security tools and processes around credit card data. However, a company may also it has satisfy the spirit of the requirements in an alternate way, if it can demonstrate that the concern has been addressed. We will focus on the requirements outlined in sections 6.6 & 11.3, but will refer to sections 10 and compensating controls as well.

Recommendations: Our strategy focuses on education and process modifications to bring security into the development lifecycle. Additionally, we suggest assessment or penetration testing services to quickly identify areas of concern. Deploy WAF to address the PCI requirement immediately. Focus on the requirements to start, but plan for a more general program, and use compensating controls as your organization evolves. Use outside help and education to address immediate gaps, both in PCI compliance and more general application security.

Training, Education, and Process Improvements: Once again, we are hammering on education and training for the development team, including project management and quality assurance. While it takes time to come up to speed, awareness by developers helps keep security issues out of the code, and is cost-effective for securing the applications. Altering the process to accommodate fixing the code is essentially free, and code improvements become part of day to day efforts. With a small code base, education and training are easy ways to reap significant benefits as the company and code base grow.

External Help: Make friends with an auditor, or hire one as a consultant to help prepare and navigate the standard. While this is not a specific recommendation for any single requirement in PCI; auditors provide an expert perspective, help address some of the ambiguity in the standard, and assist in strategy and trade-off evaluations to avoid costly missteps.

Section 11.3.2: Section 11.3 mandates penetration testing of the network and the web application. In this case we recommend external penetration testing as an independent examination of the code. It is easy to recommend penetration testing, and not because it is required in the DSS specification, rather the independent & expert review of your application behavior will closely mimic the approach hackers will take. We also anticipate budget will require you make a choice between WAF and code reviews in section 6.6, so this will provide the necessary coverage. Should you use source code reviews, one could argue that acts as a compensating control for this section, but our recommendation is to stick with the external penetration testing. External testers provide much more than just a list of specific flaws, but also identify risky or questionable application behaviors in a near production environment.

Section 6.6: Our biggest debate internally was whether to recommend Web Application Firewall or expert code review to address section 6.6 of the PCI specification. The PCI Security Standards Council recommends that you do both, but it is widely recognized that this is prohibitively expensive. WAF provides a way to quickly meet the letter of 6.6’s requirement, if not in spirit, provides basic monitoring, and is a flexible platform to block future attacks. The counter-arguments are significant and include cost, work required to customize policies for the application, and false positives & negatives. Alternatively, a code review by qualified security experts can identify weaknesses in application design and code usage and assist in education of the development team by pointing out specific flaws. Outside review is a very quick way to assess where you are and what you need. Down sides of review include cost, time to identify and fix errors, and that a constantly changing code base presents a moving target and thus requires repeated examinations.

Our recommendation here is deploy a WAF solution. Engaging a team of security professionals to review the code is an effective way to identify issues, but much of the value overlaps with the requirement of Section 11.3.2, periodic penetration testing of the application. The time to fix identified issues (even with a small-to-average body of code), with a development organization which is just coming to terms with security issues, is too long to meet PCI requirements in a timely fashion. Note that this recommendation is specific to this particular fictitious case- in other PCI audit scenarios, with a more experienced staff or a better handle on code quality, we might have made a different recommendation.

Monitoring: Database Activity Monitoring (DAM) is a good choice for Section 10 compliance- specifically by monitoring all access to credit card data. Web applications use a relational database back end to store credit card numbers, transactions, and card related data. DAM products that capture all network and console activity on the database platform provide a focused and cost-effective audit for all access to cardholder data. Consider this option for providing an audit trail for auditors and security personnel.

Internal Web Application Development


Our last use case is an internal web applications that serves employees and partners within a mid-to-large business. While this may not sound like a serious problem, given that companies have on average 1 internal application (web and traditional client/server) per 100 employees, even mid-sized companies have incredible exposure in this area. Using data from workflow, HR, accounting, business intelligence, sales, and other critical IT systems, these internal applications support employees and partners alike. And with access to pretty much all of the data within the company, security and integrity are major concerns. A common assumption is that these systems, behind the perimeter firewall, are not exposed to the same types of attacks as typical web applications, but this assumption has proven disastrous in many cases.

Investment here is motivated by fraud reduction, breach remediation, and avoidance of notification costs- and possibly by compliance. You may find this is difficult to sell to executive management if there is not a compliance mandate and hasn’t been a previous breach, but if basic perimeter security is breached these applications need some degree of resiliency rather than blind confidence in network security and access control. TJ Maxx (http://www.tjx.com/) is an excellent illustration of the danger.

Strategy: Determine basic security of the internal application, fix serious issues, and leverage education, training, and process improvements to steadily improve the quality of code. We will assume that budgeting for security in this context is far smaller than for external-facing systems, so look to cooperate between groups and leverage tools and experience.

Vulnerability Assessment and Penetration Testing: Scanning web applications for significant security, patch and configuration issues is a recommended first step in determining if there are glaring issues. Assessment tools are a cost-effective way to establish baseline security and ensure adherence to minimum best practices. Internal penetration testing will help determine the overall potential risk and prioritization, but be extremely cautious of testing live applications.

Training, Education, and Process Improvements: These may be even more important in this scenario than in our other use cases, where the business justification provides incentive to invest in security, internal web applications may not get the same degree of attention. For these applications that have a captive audience, developers have greater controls over the types of environments that they support what can be required in terms of authentication. Use these freedoms to your advantage. Training should focus on common vulnerabilities within the application stack that is being used, and give critical errors the same attention that top priority bugs would receive. Verify that these issues are tested, either as part of the VA sweep, or a a component of regression testing.

Monitoring: Monitoring for suspicious activity and system misuse is a cost-effective way to detect issues and react to them. We find WAF solutions are often too expensive for deployment across hundreds of internal applications distributed across a company, and a more cost-effective approach to collecting and analyzing activity is highly recommended. Monitoring software that plugs into the web application is often very effective for delivering some intelligence at low cost, but the burden of analyzing the data then falls on development team members. Database Activity Monitoring can effectively focus on critical information at the back end and is more mature than Web Application Monitoring.

This segment of the series took much longer to write than we originally anticipated, as our research gave us conflicting answers to some questions, making our choices were far from easy. Our recommendations really depend upon the specifics of the situation and the organization. We approached this with use cases to demonstrate how the motivating factors; combined with the current state of web security; really guides the selection of tools, services, and process changes.

We found in every case that security as part of the overall development process is the most cost-effective and least disruptive to normal operations, and is our principal recommendation for each scenario. However, as transformation of a web application does not happen overnight, we rarely have the luxury of waiting for the development team to address all security issues is not realistic; in the meantime, external third-party services and products are invaluable for dealing with immediate security challenges.

–Adrian Lane

Monday, December 29, 2008

Building A Web Application Security Program: Part 7, Secure Operations

By Adrian Lane

We’ve been covering a heck of a lot of territory in our series on Building a Web Application Security Program (see Part 1, Part 2, Part 3, Part 4, Part 5, and Part 6). So far we’ve covered secure development and secure deployment, now it’s time to move on to secure operations. This is the point where the application moves out of development and testing and into production.

Keep in mind that much of what we’ve talked about until now is still in full effect- just because you have a production system doesn’t mean you throw away all your other tools and processes. Updates still need to go through secure development, systems and applications are still subject to vulnerability assessments and penetration testing (although you need to use a different process when testing live applications vs. staging), and configuration management and ongoing secure management are more important than ever before.

In the secure operations phase we add two new technology categories to support two additional processes- Web Application Firewalls (WAF) for shielding from certain types of attacks, and monitoring at the application and database levels to support auditing and security alerting.

Before we dig in, we also want to thank everyone who has been commenting on this series as we post it- the feedback is invaluable, and we’re going to make sure everyone is credited once we put it into whitepaper format.

Web Application Firewalls (WAF)

The role of a web application firewall is to sit in front of or next to a web application, monitoring application activity, and alerting or blocking on policy violations. Thus it potentially serves two functions- as a detective control for monitoring web activity, and as a preventative control for blocking activity.

A web application firewall is a firewall specifically built to watch HTTP requests and block those that are malicious or don’t comply with specific rules. The intention is to catch SQL injection, Cross Site Scripting (XSS), directory traversal, and various HTTP abuses, as well as authorization, request forgeries, and other attempts to alter web application behavior. The WAF rules and policies are effectively consistency checks, for both the HTTP protocol and application functionality. WAFs can alert or block activity based on general attack signatures (such as a known SQL injection attack for a particular database), or application-specific signatures for the web application being protected.

WAF products examine inbound and outbound HTTP requests, compare these with the firewall rules, and create alerts for conditions of concern. Finally, the WAF selects a disposition for the traffic: 1) let it pass, 2) let it pass but audit, 3) block the transaction, or 4) reset the connection.

WAFs typically network appliances. They are normally placed in-line as a filter for the application (proxy mode); or ‘out-of-band’, receiving traffic from a mirror or SPAN port. In the former scenario, all inbound and outbound requests are intercepted and inspected prior to the web server receiving the request or user receiving the response, reducing load on the web application. For SSL traffic, inline WAFs also need to proxy the SSL connection from the browser so it can decrypt and inspect traffic before it reaches the web server, or after it leaves the web server for responses. In out-of-band mode, there are additional techniques to monitor the encrypted connections by placing a copy of the server certificate on the WAF, or positioning it behind an SSL concentrator. Some vendors also provide WAF capabilities via plug-ins for specific platforms, rather than through external devices.

The effectiveness of any WAF is limited by the quality of the policies it is configured to enforce. Policies are important not merely to ability to recognize and stop known/specific attacks, but also for flexibly dealing with ambiguous and unknown threat types, while keeping false positives manageable and without preventing normal transaction processing. The complexity of the web application, combined with the need for continuous policy updates, and the wide variety of deployment options to accommodate, pose a complex set of challenges for any WAF vendor. Simply dropping a WAF in front of your application and turning on all the default rules in blocking mode is a recipe for disaster. There is no way for black box to effectively understand all the intricacies of a custom application, and customization and tuning are essential for keeping false positives and negatives under control.

When deployed in monitoring mode, the WAF is used in a manner similar to an intrusion detection system (IDS). It’s set to monitor activity and generate alerts based on policy violations. This is how you’ll typically want to initially deploy the WAF, even if you plan on blocking activity later. It gives you an opportunity to tune the system and better understand application activity before you start trying to block connections. An advantage of monitoring mode is that you can watch for a wider range of potential attacks without worrying that false positives will result in inappropriate blocking. The disadvantages are 1) your incident handlers will spend more time dealing with these incidents and false positives, and 2) bad activity won’t be blocked immediately.

In blocking/enforcement mode, the WAF will break connections by dropping them (proxy mode) or sending TCP reset packets (out of band mode) to reset the connection. The WAF can then ban the originating IP, permanently or temporarily, to stop additional attacks from that origin. Blocking mode is most effective when deployed as part of a “shield then patch” strategy to block known vulnerabilities in your application.

When a vulnerability is discovered in your application, you build a specific signature to block attacks on it and deploy that to the WAF (the “shield”). This protects your application as you go back and fix the vulnerable code, or wait for an update from your software provider (the “patch”). The shield then patch strategy greatly reduces potential false positives that interfere with application use and improves performance, but is only possible when you have adequate processes to detect and evaluate these vulnerabilities.

You can combine both approaches by deploying a larger signature set in monitoring mode, but only enabling a few specific policies in blocking mode.

Given these challenges, satisfaction with WAF products varies widely among security professionals who use them. While WAFs are effective against known threats, they are less capable of discovering new issues or handling questionable use cases. Some WAF products are addressing these issues by linking web application firewalls more tightly to the vulnerability assessment process and tools, as we’ll discuss in a moment.


Monitoring is primarily used for discovery, both of how an application is used by real users, and also for how it can be misused. The fundamental value of monitoring is to learn what you do not already know- this is important not only for setting up a WAF, but also for tune an application security strategy. Although WAFs provide some level of application activity monitoring, there are three additional ways to monitor web applications, each with a different perspective on application activity:

  • Network Monitoring: Monitoring network activity between the user/Internet and the web server. This category includes web application firewalls, intrusion detection systems, packet sniffers, and other external tools. While generic network security and sniffing tools can capture all network traffic, they have much less capability to place it in context and translate network activity into application transactions. Simply viewing HTTP traffic is often insufficient for understanding what users are attempting in an application- this is where interpretation is required. If a solution includes web application specific analysis and the ability to (potentially) audit all web activity, we call it Web Application Monitoring (WAM). While network monitoring is easy to implement and doesn’t require application changes, it can only monitor what’s going into and coming out of the application. This may be useful for detecting traditional attacks against the application stack, but much less useful than seeing traffic fully correlated to higher-level transactions.
  • Application Auditing/Logging: Collection and analysis of application logs and internal activity. Both custom and off-the-shelf applications often include internal auditing capabilities, but there is tremendous variation in what’s captured and how it’s formatted and stored. While you gain insight into what’s actually occurring in the application, not all applications log equally (or at all)- you are limited to whatever the programmers decided to track. For major enterprise applications, such as SAP, we’ve seen third party tools that either add additional monitoring or can interpret native audits. Log management and SIEM tools can also be used to collect and interpret (to a more limited degree) application activity when audit logs are generated.
  • Database Activity Monitoring: DAM tools use a variety of methods to (potentially) record all database transactions and generate alerts on specific policy violations. By monitoring activity between the application and the database, DAM can provide a more precise examination of data elements, and awareness of multi-step transactions which directly correspond to business functions. Some DAM tools have specific plug ins for major application types (e.g., SAP & PeopleSoft) to translate database transactions into application activity. Since the vast majority of web applications run off databases, this is an effective point to track activity and look for policy violations. A full discussion of DAM is beyond the scope of this post, but more information is available in our DAM whitepaper.

WAF can be used for monitoring, but these alternative tools offer a wider range of activity collection, which helps to detect probing which may not be obvious from HTTP requests alone. The focus of web application monitoring is to examine behavior across data sources and provide analysis, recording activity trails and alerting on suspicious events, whereas WAFs are more focused on detection and blocking of known threats. There are significant differences between WAMs and WAFs in areas such as activity storage, aggregation of data from multiple sources, and examination of the collected data, so choosing the best tool depends specifics of the requirement. We must point out that web application monitoring products are not fully mature, and the handful of available products are early in the product evolution cycle.


Several vendors have begun providing a hybrid model that combines web application vulnerability assessment with a web application firewall. As mentioned earlier, one of the difficulties with a shield then patch strategy is detecting vulnerabilities and building the WAF signatures to defend them. Coupling assessment tools or services with WAFs by feeding the assessment results to the firewall, and having the firewall adjust its policy set accordingly, can makes the firewall more effective. The intention is to fill the gap between exploits discovery/announcement and deployment of tested patches in the application, by instructing the WAF to block access to the particular weakness. In this model the assessment determines that there is a vulnerability and feeds the information to the WAF. The assessment policy contains WAF instructions on what to look for and how to respond. The WAF then dynamically incorporates the policy and protects the application.

This is the last post in this series where we discuss your options at different stages of the application lifecycle. Our next post will discuss which options you should consider, and how to balance your resource expenditures into an overall program.

–Adrian Lane

Tuesday, December 16, 2008

Building a Web Application Security Program: Part 6, Secure Deployment

By Rich

In our last episode, we continued our series on building a web application security program by looking at the secure development stage (see also Part 1, Part 2, Part 3, Part 4, and Part 5).

Today we’re going to transition into the secure deployment stage and talk about vulnerability assessments and penetration testing. Keep in mind that we look at web application security as an ongoing, and overlapping, process. Although we’ve divided things up into phases to facilitate our discussion, that doesn’t mean there are hard and fast lines slicing things up. For example, you’ll likely continue using dynamic analysis tools in the deployment stage, and will definitely use vulnerability assessments and penetration testing in the operations phase.

We’ve also been getting some great feedback in the comments, and will be incorporating it into the final paper (which will be posted here for free). We’ve decided this feedback is so good that we’re going to start crediting anyone who leaves comments that result in changes to the content (with permission, of course). It’s not as good as paying you, but it’s the best we can do with the current business model (for now- don’t assume we aren’t thinking about it).

As we dig into this keep in mind that we’re showing you the big picture and everything that’s available. When we close the series we’ll talk prioritization and where to focus your efforts for those of you on a limited budget- it’s not like we’re so naive as to think all of you can afford everything on the market.

Vulnerability Assessment

In a vulnerability assessment we scan a web application to identify anything an attacker could potentially use against us (some assessments also look for compliance/configuration/standards issues, but the main goal in a VA is security). We can do this with a tool, service, or combination of approaches.

A web application vulnerability assessment is very different than a general vulnerability assessment where we focus on network and hosts. In those, we scan ports, connect to services, and use other techniques to gather information revealing the patch levels, configurations, and potential exposures of our infrastructure. Since, as we’ve discussed, even “standard” web applications are essentially all custom, we need to dig a little deeper, examine application function and logic, and use more customized assessments to determine if a web application is vulnerable. With so much custom code and implementation, we have to rely less on known patch levels and configurations, and more on actually banging away on the application and testing attack pathways. As we’ve said before, custom code equals custom vulnerabilities. (For an excellent overview of web application vulnerability layers please see this post by Jeremiah Grossman. We are focusing on the top three layers- third-party web applications, and the technical and business logic flaws of custom applications).

The web application vulnerability assessment market includes both tools and services. Even if you decide to go the tool route, it’s absolutely critical that you place the tools in the hands of an experienced operator who will understand and be able to act on the results. It’s also important to run both credentialed and uncredentialed assessments. In a credentialed assessment, the tool or assessor has usernames and passwords of various levels to access the application. This allows them inside access to assess the application as if they were an authorized user attempting to exceed authority.


There are a number of commercial, free, and open source tools available for assessing web application vulnerabilities, each with varying capabilities. Some tools only focus on a few kinds of exploits, and experienced assessors use a collection of tools and manual techniques. For example, there are tools that focus exclusively on finding and testing SQL injection attacks. Enterprise-class tools are broader, and should include a wide range of tests for major web application vulnerability classes, such as SQL injection, cross site scripting, and directory traversals. The OWASP Top 10 is a good starting list of major vulnerabilities, but an enterprise class tool shouldn’t limit itself to just one list or category of vulnerabilities. An enterprise tool should also be capable of scanning multiple applications, tracking results over time, providing robust reporting (especially compliance reports), and providing reports customized local needs (e.g., add/drop scans).

Tools are typically software, but can also include dedicated appliances. Tools can run either manual scans with an operator behind them, or automatics scans for on a schedule. Since web applications change so often, it’s important to scan any modifications or new applications before deployment, as well as live applications on an ongoing basis.


Not all organizations have the resources or need to buy and deploy tools to assess their own applications, and in some cases external assessments may be required for compliance.

There are three main categories of web application vulnerability assessment services:

  • Fully automatic scans: These are machine-run automatic scans that don’t involve a human operator on the other side. The cost is low, but they are more prone to false positives and false negatives. They work well for ongoing assessments on a continuous basis, but due to their limitations you’ll likely still want more in-depth assessments from time to time.
  • Automatic scans with manual evaluation: An automatic tool performs the bulk of the assessment, followed by human evaluation of the results and additional testing. They provide a good balance between ongoing assessments and the costs of a completely manual assessment. You get deeper coverage and more accurate results, but at a higher cost.
  • Manual assessments: A trained security assessor manually evaluates your web application to identify vulnerabilities. Typically an assessor uses their own tools, then validates the results and provides custom reports. The cost is higher per assessment than the other options, but a good assessor may find more flaws.

Penetration Testing

The goal of a vulnerability assessment is to find potential avenues an attacker can exploit, while a penetration test goes a step further and validates whether attack pathways result in risk to the organization. In a web application penetration test we attempt to penetrate our own applications and determine what an attacker can do, and what the consequences might be.

Vulnerability assessments and penetration tests are highly complementary, and frequently performed together since the first step in any attack is to find vulnerabilities to exploit. The goal during the vulnerability assessment phase is to find as many of those flaws as possible, and during the penetration test to validate those flaws, determine potential damages, and prioritize remediation efforts.

That’s the key value of a penetration test- it bridges the gap between the discovered vulnerability and the exploitable asset so you can make an appropriate risk decision. We don’t just know we’re vulnerable, we learn the potential consequences of those vulnerabilities. For example, your vulnerability scan may show a SQL injection vulnerability, but when you attempt to exploit it in the penetration test it doesn’t reveal sensitive information, and can’t be used to damage the web application. On the other hand, a seemingly minor vulnerability might turn out to allow exploitation of your entire web application.

Some experts consider penetration tests important because they best replicate the techniques and goals an attacker will use to compromise an application, but we find that a structured penetration test is more valuable as a risk prioritization tool. If we think in terms of risk models, a properly performed penetration test helps fill in the potential severity/impact side of the analysis. Of course, this also means you need to focus your penetration testing program on risk assessment, rather than simply “breaking” a web application.

As with vulnerability assessments, penetration tests can include the entire vulnerability stack (from the network and operating system up through your custom application code), and both tools and services are available. Again, we’ll limit this discussion to web application specific aspects of penetration testing.


A web application penetration testing tool provides a framework for identifying and safely exploiting web vulnerabilities, and measuring or estimating their potential impact. While there are many tools used by penetration testers, most of them are point tools, rather than broad suites. An enterprise-class tool adds features to better assist the risk management process, support internal assessments, and to reduce the costs of internal penetration tests. This includes broad coverage of application vulnerability classes, “safe” techniques to exploit applications without interfering with their ongoing use, workflow, extensive reporting, and automation for ongoing assessments. One advantage of penetration testing tools is you can integrate them into the development and deployment process, rather than just assessing live web applications.

Penetration testing tools are always delivered as software and should be used by a trained operator. While parts of the process can be automated, once you dig into active exploitation and analysis of results, you need a human being involved.

When using penetration testing (or vulnerability assessment) tools, you have the choice of running them in a safe mode that reduces the likelihood of causing a service or application outage (this is true for all kinds of VA and penetration tests, not just web applications). You’ll want to use safe mode on live applications. But these tools are extremely valuable in assessing development/test environments before applications are deployed, and you shouldn’t restrict your exploit attempts to non-production systems.


Unlike vulnerability assessments, we are unaware of any automated penetration testing services (although some of the automatic/manual services come close). Due to the more intrusive and fluid nature of a web application penetration test you always need a human to drive the process. Penetration testing services are typically offered as either one-off consulting projects or a subscription service for scheduled tests. Testing frequency varies greatly based on your business needs, exposure, and the nature of your web applications. Internal applications might only be assessed annually as part of a regularly scheduled enterprise-wide test.

When evaluating a penetration testing service it’s important to understand their processes, experience, and results. Some companies offer little more than a remote scan using standard tools, and fail to provide prioritized results that are usable in your risk assessment. Others simply “break into” web applications and stop as soon as they get access. You should give preference to experienced organizations with an established process that can provide sample reports (and references) that meet your needs and expectations. Most penetration tests are structured as either fixed-time or fixed-depth. In a fixed-time engagement (the most common) the penetration testers discover as much as they can in a fixed amount of time. Engagements may also divide the time used into phases- e.g. a blind attack, a credentialed attack, and so on. Fixed-depth engagements stop when a particular penetration goal is achieved, no matter how much time it takes (e.g. administrative access to the application or access to credit card numbers).

Integrating Vulnerability Assessment and Penetration Testing into Secure Deployment

Although most organizations tend to focus web application vulnerability assessments and penetration tests on externally accessible production applications, the process really begins during deployment, and shouldn’t be limited to external applications. It’s important to test applications in depth before you expose them to either the outside world or internal users. There’s also no reason, other than resources, you can’t integrate assessments into the development process itself at major milestones.

Before deploying an application, set up your test environment so that it accurately reflects production. Everything from system configurations and patch levels, up through application connections to outside services must match production or results won’t be reliable. Then perform your vulnerability assessment and penetration test. If you use tools, you’ll do this with your own (appropriately trained) personnel. If you use a service, you’ll need to grant them remote access to your test environment. Since few organizations can afford to test every single application to the same degree, you’ll want to prioritize based on the exposure of the application, its criticality for business operations, and the sensitivity of the data it accesses.

For any major externally accessible application you’ll want to engage a third party for both VA and penetration testing before deployment. If your business relies on it for critical operations, and it’s publicly accessible, you really want to get an external assessment.

Once an application is deployed, your goal should be to perform at least a basic assessment on any major modifications before they commit. For critical applications, engage a third-party service for ongoing vulnerability assessments, and periodic penetration tests (at least annual or after major updates) on top of your own testing.

We know not all of you have the resources to support internal and external tools and services for VA and penetration testing on an ongoing basis, so you’ll need to adapt these recommendations for your own organizations. We’ve provided a high level overview of what’s possible, and some suggestions on where and how to prioritize.

In our next post we’ll close out our discussion of web application security technologies by looking at web application firewalls and monitoring tools. We’ll then close out the series by showing you how to put these pieces together into a complete program, how to prioritize what you really need, and how to fit it to the wild world of web application development.


Structured Security Program, meet Agile Process

By Adrian Lane

Bryan Sullivan’s thought-provoking post on Streamlining Security Practices for Agile Development caught my attention this morning. Reading it gave me the impression of a genuine generational divide. If you have ever witnessed a father and son talk about music, while they are talking about the same subject, there is little doubt the two are incompatible.

The post is in line with what Rich and I have been discussing with the web application series, especially in the area of why the web apps are different, albeit on a slightly more granular level. This article is about process simplification and integration, and spells out a few of the things you need to consider if moving from more formalized waterfall process into Agile with Security. The two nuggets of valuable information are the risk based inclusion of requirements, where the higher risk issues are placed into the sprints, and the second is how to account for lower priority issues that require periodic inspection within a non-linear development methodology.

The risk-based approach of placing higher security issues as code gets created in each sprint is very effective. It requires that issues and threats be classified in advance, but it makes the sprint requirements very clear while keeping security as a core function of the product. It is a strong motivator for code and test case re-use to reduce overhead during each sprint, especially in critical areas like input validation.

Bryan also discusses the difficulties of fitting other lower priority security requirements extracted from SDL into Agile for Web Development. In fact, he closes the post with the conclusion that retrofitting waterfall based approaches to secure Agile development is not a good fit. Bravo to that! This is the heart of the issue, and while the granular inclusion of high risk issues into the sprint works, the rest of the ‘mesh’ is pretty much broken.  Checks and certifications triggered upon completed milestones must be rethought. The bucketing approach can work for you, but what you label the buckets and when you give them consideration will vary from team to team. You may decide to make them simple elements of the product and sprint backlog. But that’s the great thing about process is you get to change it to however it suits your purpose.

Regardless, this post has some great food for though and is worth a read.

–Adrian Lane

Thursday, December 11, 2008

How The Cloud Destroys Everything I Love (About Web App Security)

By Rich

On Tuesday, Chris Hoff joined me to guest host the Network Security Podcast and we got into a deep discussion on cloud security. And as you know, for the past couple of weeks we’ve been building our series on web application security. This, of course, led to all sorts of impure thoughts about where things are headed. I wouldn’t say I’m ready to run around in tattered clothes screaming about the end of the Earth, but the company isn’t called Securosis just because it has a nice ring to it.

If you think about it a certain way, cloud computing just destroys everything we talk about for web application security. And not just in one of those, “oh crap, here’s one of those analysts spewing BS about something being dead” ways. Before jumping into the details, in this case I’m talking very specifically of cloud based computing infrastructure- e.g., Amazon EC2/S3. This is where we program our web applications to run on top of a cloud infrastructure, not dedicated resources in a colo or a “traditional” virtual server. I also sprinkle in cloud services- e.g., APIs we can hook into using any application, even if the app is located on our own server (e.g., Google APIs).

Stealing from our yet incomplete series on web app sec and our discussions of ADMP, here’s what I mean:

  • Secure development (somewhat) breaks: we’re now developing on a platform we can’t fully control- in a development environment we may not be able to isolate/lock down. While we should be able to do a good job with our own code, there is a high probability that the infrastructure under us can change unexpectedly. We can mitigate this risk more than some of the other ones I’ll mention- first, through SLAs with our cloud infrastructure provider, second by adjusting our development process to account for the cloud. For example, make sure you develop on the cloud (and secure as best you can) rather than completely developing in a local virtual environment that you then shift to the cloud. This clearly comes with a different set of security risks (putting development code on the Internet) that also need to be, and can be, managed. Data de-identification becomes especially important.
  • Static and dynamic analysis tools (mostly) break: We can still analyze our own source code, but once we interact with cloud based services beyond just using them as a host for a virtual machine, we lose some ability to analyze the code (anything we don’t program ourselves). Thus we lose visibility into the inner workings of any third party/SaaS APIs (authentication, presentation, and so on), and they are likely to randomly change under our feet as the providing vendor continually develops them. We can still perform external dynamic testing, but depending on the nature of the cloud infrastructure we’re using we can’t necessarily monitor the application during runtime and instrument it the same way we can in our test environments. Sure, we can mitigate all of this to some degree, especially if the cloud infrastructure service providers give us the right hooks, but I don’t hold out much hope this is at the top of their priorities. (Note for testing tools vendors- big opportunity here).
  • Vulnerability assessment and penetration testing… mostly don’t break: So maybe the cloud doesn’t destroy everything I love. This is one reason I like VA and pen testing- they never go out of style. We still lose some ability to test/attack service APIs.
  • Web application firewalls really break: We can’t really put a box we control in front of the entire cloud, can we? Unless the WAF is built into the cloud, good luck getting it to work. Cloud vendors will have to offer this as a service, or we’ll need to route traffic through our WAF before it hits the back end of the cloud, negating some of the reasons we switch to the cloud in the first place. We can mitigate some of this through either the traffic routing option, virtual WAFs built into our cloud deployment (we need new products for it), or cloud providers building WAF functionality into their infrastructure for us.
  • Application and Database Activity Monitoring break: We can no longer use external monitoring devices or services, and have to integrate any monitoring into our cloud-based application. As with pretty much all of this list it’s not an impossible problem, just one people will ignore. For example, I highly doubt most of the database activity monitoring techniques will work in the cloud- network monitoring, memory monitoring, or kernel extensions. Native audit might, but not all database management systems provide effective audit logs, and you still need a way to collect them as your app and db shoot around the cloud for resource optimization.

I could write more about each of these areas, but you get the point. When we run web applications on cloud based infrastructure, using cloud based software services, we break much of the nascent web application security models we’re just starting to get our fingers around. The world isn’t over*, but it sure just moved out from under our feet.

*This doesn’t destroy the world, but it’s quite possible that the Keanu Reeves version of The Day the Earth Stood Still will.