The Good (Yes, Good) And Bad Of PCI

Written by rmogull | Filed under Misc, Non-Geek, Risk | 20 comments

20 comments

1. Andre Gironda Jun 3

   WAF isn’t always a quick-fix, nor is secure coding only long-term. There are shades of gray, such as operational issues with WAF’s (e.g. false-positives, blocking mode, performance issues, the issue that most solutions are input validation and not input/output sanitation, etc) and short-term code fixes such as aspect-oriented programming point-cuts.

   Finally, even if WAF or secure coding is used — there is the issue with regression testing. Every new release may require both a code and/or WAF adjustment due to the reintroduction of old defects. The easiest way to solve the regression testing problem is to utilize continuous-prevention development, which often requires continuous integration and test-first development (where a unit test or in-container test asserts the defect’s fix instead of just the defect).

   What PCI-DSS attempts to do is improve our situation by bringing awareness to web application security issues in Requirement 6.6. It’s not a solution/prevention check-box, it’s an awareness/bullshit-detector check-box. Many organizations do not have metrics on software security issues (although some might have metrics on application security issues) so whether a WAF will work or not for the most critical risk(s) to the payment application becomes an issue dependent on specific factors unique to that organzation’s application deployment/infrastructure/etc.

   What PCI-DSS gets completely wrong is that WAF’s and black-box scanners cannot “prevent OWASP Top 10″. OWASP Top 10 is based on MITRE CWE, and the only way to prevent MITRE CWE (and therefore OWASP Top 10) is to code around software weaknesses. How would a WAF or black-box scanner find/prevent problems with cryptographic storage? With domain logic?

   There is also a difference between the attack surface of web applications versus fat applications. Fat applications often suffer from big, obvious buffer overflows, which are activation-dependent and can be largely protected by a single exploitation countermeasure such as ASLR. Web applications suffer from a variety of small, non-obvious weaknesses. When these small and numerous weaknesses are combined, protection mechanisms fall the wayside. Exposure of source code or error handling may open a new door to a totally different attack. Second-order SQLi can only be prevented by proper use of parameterized queries (not input validation alone). HTML/CSS/JS injection (XSS and look-and-feel hacks) can only be prevented by proper use of input/output sanitation plus dealing with canocalization as best as possible (not input validation alone). CSRF protection is dependent on HTMl/CSS/JS injection protection. This list can go on forever.

2. rmogull Jun 3

   Dre,

   In some ways this is a response to your other comment on the WAF post, as well as this post.

   I’m not going to argue with wat you say at all, but the truth is nearly no organization has the resources to adopt the processes you are recommending. As much as I hate to say it, and as much as we need to drive towards being more process driven, we’re possibly decades from achieving that goal on a broad basis.

   Yes, these are all shades of gray, but there’s only so much I can stuff into some blog posts and the occasional paper, and we’re often hunting for the biggest hits with the lowest effort. My personal goal with all of these is to slowly drive people in the right direction with targeted tactical deployments to block out as much attack surface as possible, in the sortest time possible.

   Truth be told, I think driving forward with tools and ADMP will buy us more, faster, than reworking processes with a user base that lacks the skills, expertise, and resources they really need to get the job done.

   It’s shield and patch- but patching means changing the entire way we develop and manage web applications, not just fixing a few bugs. We’ll shield as best we can knowing it doesn’t fix the root problem, but it buys us the time for the more intensive work we also need.

3. Andre Gironda Jun 3

   @ Rich,

   You said, “nearly no organization has the resources to adopt the processes you are recommending”

   This is an overgeneralization. Organizations have corresponding maturity levels for infrastructure/operations and software development. I like to use Gartner IMM for infrastructure/operations and CMMI for software development, although certainly there are other models, and ones that are even security-specific (ISM3).

   Some organizations may rate higher on CMMI than IMM. In this case, it might be easier to adopt some of my suggested processes for SDLC rather than a new operations policy for WAF deployment/administration.

   You also said, “we’re often hunting for the biggest hits with the lowest effort. My personal goal with all of these is to slowly drive people in the right direction with targeted tactical deployments to block out as much attack surface as possible, in the shortest time possible”

   Secure SDLC solutions such as the ones that I have recommended block out more attack surface and can take a much shorter time to implement versus ADMP/tools. I’m all about biggest hit / lowest effort, hence my recommendations.

   I have several canned methods that make continuous integration improvements a snap — something that can be done overnight. More often than not lead developers know about the problems in the software they produce, but are not empowered to do anything about it. In other words, the people solutions are already usually there (or they are within easy reach). It is the process problems that must be solved, and because technology solutions are dependent on process, it’s the only way. There are no shortcuts. There is no way to “buy time”. The cake is a lie. I have seen WAF/IPS/IDS epically fail time and time again, from day one to day one-thousand. Where did anyone see these technologies succeed? Did they / can they stop modern attacks such as Danmec/Asprox?

4. Amichai Shulman Jun 3

   Must agree on this one with Rich, most organizations do not have resources or are not built in a way that would allow putting stringent security into development process. Also I think that PCI were indeed wrong to begin with putting WAF and Code Review one against the other. However, given the role of 6.6 as a deployment time I clearly think that WAF is the better solution (proper disclosure - I work for a WAF vendor) since it brings to the table the ability not only to know about a vulnerability but also to mitigate it. See my byline on this at http://www.technewsworld.com/story/62701.html

   Cheers,
   Amichai

5. Andre Gironda Jun 4

   @ Amichai

   “most organizations”

   I would be curious as to who these mystical “most” organizations are. Where are you getting your numbers from? I’ve queried the Global 2k, Fortune 50/100/500/1k/2k, any vertical that I can think of — as well as small-medium business, education, healthcare, et al. All of these can integrate some small changes in their development process that translate into huge wins for mitigation of major risks. Note that this has nothing to do with manual or automated code reviews.

   Notice where I use the words, `continuous integration’ and `test-first development’ and not `code review’. You can’t understand the intricacies to CI or TDD from the Wikipedia articles, you’ll have to see them work with real dev teams. You’ll have to be creative when integrating secure development practices along with these concepts.

   Oh wait, you work for a WAF vendor. Nevermind. See:
   http://securitybuddha.com/2008/05/24/presenting-security-ideas-or-driving-agendas/

6. rmogull Jun 4

   Dre,

   Pull the claws back, we’re just having a civil discussion and he didn’t hide who he works for.

   Now I clearly get to pull the “most organizations” card, since I’ve probably talked to more of them than you have. Here’s my response.

   First, please remember I’m only talking about *web applications* not traditional apps. The environment is definitely different everywhere I’ve looked between the two, with very few exceptions.

   1) Can most organizations make small changes with big impact? Yes. Absolutely.
   2) Will these be enough? No- we need more on top of that.
   3) Will it help with all the legacy code? No.
   4) Will it help with machine/tool generated code, often from major back-end applications? Mostly no, rarely yes.
   5) Will it help with third-party/SI/independent code? No.
   6) Will it help shield against newly discovered vuln classes? No.
   7) Is there high turnover in most of these organizations among web app developers? Yes.
   8) Are they trained as well as traditional developers on processes and fundamental skills? Rarely.
   9) Will WAFs stop everything? No- they are to immediately shield to buy us the time to fix.
   10) Will ADMP stop everything? No, but it will work a lot better than WAFs.
   11) Will people continue to make even simple coding errors after we’ve trained them and made process changes? Yes.

7. Sharon Besser Jun 4

   All

   I like the attitude of keeping the discussion calm. For the sake of full disclosure, I’d like to mention that I’m working for an Application Data Security company, with Amichai (yes, WAF is one product we develop and sell).

   Rich,
   I agree with your comments. That’s what our customers and prospects tell us, but also what we learned from other industry experts. Very recently I wrote about it here: http://blog.imperva.com/2008/06/how-long-does-it-take-a-lot-sa.html (quoting Jeremiah Grossman) and here: http://blog.imperva.com/2008/05/giving-sql-injection-the-respe.html(quoting Microsoft research)

8. Andre Gironda Jun 4

   @ Rich

   I can’t pull my claws back on this issue. Imperva may say, “WAF now; patch later”, but what people hear is “Spend a lot of money with Imperva to solve all your problems and forget about ever patching or improving SDLC process”.

   I’m glad that we have set the scope to web applications. This was my intention. WAF’s don’t protect fat applications.

   Your numbered list of 1 to 11 is amusing. AOP point-cuts, for example, work with legacy, code generated, and third-party code.

   I don’t understand this WAF argument that says that WAF protects against new vulnerability classes (and that secure coding practices do not). This argument is a fallacy.

   Sure, there might be high-turnover in web application developers, but I think there is also high turnover in operations departments.

   Also — it appears that you do not understand the distinction between front-end programmers and back-end programmers. Often, front-end programmers only deal with HTML/CSS. There are plenty of resources (such as the book `Refactoring HTML’, which I’m listed in the praise section for!) for front-end programmers, but even shops that don’t make these types of changes will only output one major defect: lack of validated and well-formed XHTML. This is not really a huge issue compared to say: output encoding, parameterized queries, issues in the domain logic, et al.

   “Are they trained as well as traditional developers on processes and fundamental skills?” “Will people continue to make even simple coding errors after we’ve trained them and made process changes?”

   Excellent questions! Now you’re down the right track. How do we deal with developers, how do we give them good process to work behind, and fundamental skills to understand process with security/modern-risk added? How do we train them enough / how do we add enough process to get them to create software with less critical software vulnerabilities?

   @ Sharon

   We can have a calm discussion about WAF when you stop developing and selling them. Until then, it’s “game on” because I hate to see people wasting money on these solutions. It’s like AV, firewall, and IPS all over again, and I would rather educate than suffocate.

   Sharon, would you consider me to be an industry expert? Citing one industry expert over and over again is often a bad way to represent your data. The truth is that because you develop/sell WAF’s, you will never see my perspective. You will always believe what I say to be a lie and cover it up as best as possible.

   My research proves Jeremiah Grossman wrong. My research says that the Microsoft SDL is the wrong way to develop modern web applications. I know this it is hard to believe that a no-namer such as myself could do better than what WhiteHatSec and Micrsoft have done. But what have they done? What has Imperva done?

   However, I am no longer alone. Dave Wichers of Aspect Security recently presented on “Fundamental Application Security Building Blocks - The Benefits of Establishing an Enterprise Security API (ESAPI) for Your Organization” and “Agile Security - Breaking the Waterfall Mindset of the Security Industry” at the OWASP AppSec EU Conference in Belgium two weeks ago. These two presentations demonstrate the future of web application software development technology and process. Cigital, the Department of Homeland Security’s Build-Security-In web portal, and the TS/SCI Security Blog team have also made similar improvements over the past 2-3 years.

   In fact, Microsoft itself does indeed speak to software development lifecycle process for web applications (but only for ASP.NET with either VB.NET or C# .NET). They have an exam program, courseware, a book, and everything. Yes, it’s completely different than the Microsoft SDL. It’s their “Implementing Security for Applications with Microsoft Visual C# .NET” (and VB) — exams 70-330 and 70-340. The course is a 5-day instructor-led training called “Implementing Security for Applications”, Course 2840. The book (August, 2004) has the same name as the course as a self-paced training kit.

   In this material, chapters 1-3 promote secure coding best-practices, test-first development, and the use of static checkers along with a process that addresses design for test and design for security issues. The training kit even includes a CDROM that helps developers learn how to perform proper unit testing in a test-first paradigm to test for security properties using NUnit/NUnitASP/TestRunner.

   Certainly, Microsoft has made strides since 2004, and the material could use an update. For example, FxCop’s security checking capabilities are about to be separated into two new tools: CAT.NET (the free XSSDetect utility is a part of Microsoft’s internal CAT.NET which is about to be released free to community — probably sometime this year) and StyleCop (a classic static checker such as PMD for Java or lint for C). I’m sure FxCop will still be a valid tool (as mentioned in Implementing Security for Applications), but it’s great to see tool improvements. FxCop isn’t perfect, and StyleCop, XSSDetect, and CAT.NET provide more tools that work better under different scenarios.

   Microsoft is also changing the way that ASP.NET 3.x works. They have added support for the MVC pattern, and therefore, AOP and DI. Their Dependency Injection (DI) framework is known as Unity. Surely, ASP.NET MVC is cutting-edge — so it doesn’t apply to legacy ASP.NET 1.0/1.1/2.x code and we’re not going to see the benefits until much later. However, ASP.NET is not new to the concepts of TDD, AOP, or DI. NUnit has been around for a very long time, and ASP.NET 1.0-2.0 can utilize free code coverage tools such as NCover. ASP.NET 3.x must use a newer free tool called PartCover (and NUnitASP is no longer being developed, but something else will eventually replace it). AOP and DI were possible with ASP.NET 2.0 using AspectDNG and MonoRail (MVC for ASP.NET 2.0), respectively.

   Yes, this doesn’t apply to all dev shops. However, I have little things like this that apply to dev shops, frameworks, and programming langauges of all types. I have yet to run across a dev shop that can’t make small improvements such as these which cost nothing (and often improve their current situation much more than any other technique). Selling or promoting WAF is killing these activities.

   To be honest, if I had the AIDS cure — I’m sure that pharmaceutical companies would be pretty pissed off at me. This is what it feels like to fight WAF and web application scanner technology everyday.

9. Max Jun 9

   Rich,

   I was wondering when you’d finally glom onto the goodness that PCI can bring. Here in our shop (a Tier-1 merchant) we’re in the midst of PCI and we’ve seen a large amount of investment in new systems and in updated and improved policy and procedure to get compliant. Far and away the re-work of stale policies has the best chance of helping us out.

   Two quick comments:

   1) I think a lot of the goodness we’ve seen is because (finally) someone other than the security guys have told the business they need to do something. I think that in many quarters the security teams have been seen as crying wolf for too long and are now on perma-ignore with the business.

   2) I think a lot of the PCI-hatin’ that goes on is because some folks thought PCI was a silver bullet fix to all things security. It’s not. At best PCI helps an enterprise start becoming directionally correct and, at worst, a paper-filling-out exercise. Any standard, product, or technology can be nit-picked until it appears useless…but that doesn’t mean you should do nothing.

   Lastly, on the WAF vs. Code Review issue above, methinks that there are a lot of wasted elecrons there. Code review can work great for some stuff and WAF can work great for other stuff. Neither is a silver bullet.

   Again, there are no 100% solutions anywhere.

   Cheers,

   Max

10. Gary Jun 22

    What is it with silver bullets?

    The only people who mention them do so only to say they don’t exist … as if we thought they do.

    I’ve never met anyone (no, not even vendors) who claim to have or sell silver bullets. I don’t even know of anyone who is searching for silver bullets.

    I’ve seen silver bullets in old Dracula films, but that’s nothing to do with information security, and is clearly pure fantasy.

    Nobody believes in silver bullets. Telling me they don’t exist merely demonstrates your ignorance and naivete.

    :-)

11. rmogull Jun 23

    1) I never use the words “silver bullet” in thie post, so not sure of the reference.
    1) Plenty of people ask for them or claim to sell them, every single day. Except they usually call them, “black boxes”. Usually it’s people that don’t understand the issues, because it’s outside of their domain expertise. Can’t blame them for that if their job is something else.
    3) Or were you picking on Andre? I’m totally okay with that :)

12. Anonymous Jun 23

    @ rmogull:

    Gary was picking on Max, not me. Nobody picks on me and gets away scot-free. I’ll keep trolling their blog until they go crazy. I have bots to help me and everything.

    Also, on a very happy note, I really do think that Imperva SecureSphere is not only a silver bullet, but it’s also a magical silver bullet. In fact, I’d go as far to say that not using Imperva SecureSphere should be illegal, wouldn’t you?

    I mean, it takes at least a year or two to fix any given SQL injection vulnerability, and developers are stupid, lazy, and annoying. Developers will never understand security, so we have to implement security IN THE NETWORK. The network is the computer, and network security is a magical silver bullet.

    Note: someone on the Internet will fail to find any of the sarcasm in the above.

13. rmogull Jun 23

    Dre,

    I’ll continue to disagree that we have the time or resources to fix all of those vulns quickly enough that we don’t need to put something in from of the web apps.

    I used to think we could solve this all with good code development practices, but not any more for the reasons I outlined above. We need both.

14. Andre Gironda Jun 23

    @ rmogull:

    I’ll continue to disagree that we have the time or resources to implement WAF to block any of those vulns quickly enough that we could have spent more time and resources on identification of root-cause and steps towards a better, more secure SDLC.

    How long does it typically take to implement a WAF? Which vulnerabilities do they protect against? I think I know these answers. Do you?

    We don’t need both WAF and secure coding… this is not correct.

15. rmogull Jun 23

    Dre,

    We’re never going to agree on this. Between our in-person and online discussions we’ve spent hours on it now.

    Today’s WAFs are not the long term solution; as I’ve written before they need serious help. By the same token, I’m absolutely convinced we can’t just solve this with a better SDLC. Too many old vulenrabilities, too much machine generated code, too many chances for human error.

    My vote is firmly in shield then patch- preferably with the next generation web applciation anti-exploitation tools I call ADMP. If we were having this argument 10 years ago (more like 12-13) I might agree with you, but it’s too late.

    The operational realities of the dozens to hundreds of end user organizations I talk to every year are just too down and dirty for what you’re suggesting.

16. Andre Gironda Jun 23

    @ rmogull:

    Yes, but I’m convinced that I’ll convince you otherwise.

    I’m going to give ADMP and WAF some breaks once I see that they are doing the right things. As an example, see our latest post on WAF egress technology. However, the way that WAF and ADMP work today is a complete waste of money and time. If you want to talk specific products, we can really get into it — but let’s do that offline.

    However, “shield and patch” is really just the wrong way to say it in the first place. If you want to prevent attacks, you want to prevent them from exiting your network (like DLP) — not entering (like a shield). Re-coding applications is really about Refactoring, proper use of AOP or DI as well as interative programming (often seen in newer paradigms as XP, Agile, SCRUM, CI, and/or TDD). There is nothing “patch”-like about it.

    I have no intention of talking about old vulnerabilities or times past unless they are relevant to attacks today. Attacks which cause data breaches; attacks which cause turmoil.

    “The operational realities of the dozens to hundreds of end user organizations I talk to every year are just too down and dirty for what you’re suggesting” — Honestly, I don’t see how the same could not be said about WAF or ADMP. Do they or don’t they have operational problems? How easy is it just to drop WAF or ADMP into an existing architecture?

    Let me add one more question to my list of questions that you haven’t yet answered: “How long does it typically take to implement a WAF? Which vulnerabilities do they protect against?” — “Which software weaknesses does WAF/ADMP fail to protect against?”

17. Andre Gironda Jun 23

    Oops, wrong link (although that’s also a good one). Check out our latest post (Marcin wrote it up based on real-world experience with the very products that we normally talk about):

    Web application firewalls: A slight change of heart

    TS/SCI Security Blog

18. Mike Jun 29

    Controversy will get you readers, but not the truth. Seek first to understand and then to be understood.

19. rmogull Jun 30

    Mike, aside from being pretentious, that’s just illinformed for someone running a PCI site (pcianswers.com). I’ve worked with hundreds of organizations dealing with PCI.

    Besides, speaking of truth aren’t you the one that posted on Scanless PCI, realized you were made a fool of, and deleted your post instead of issuing a retraction/correction?

    Just wondering.

1. What we’re reading, week of 6/9 « VPN Haus

Leave a reply