I am a big fan of the Agile project development methodology, especially Agile with Scrum. I love the granularity and focus the approach requires. I love that at any given point in time you are working on the most important feature or function. I love the derivative value of communication and subtle form of peer pressure that Scrum meetings produce. I love that if mistakes are made you do not go too far in the wrong direction, resulting in higher productivity and few software projects that are total disasters. I think Agile is the biggest advancement in code development in the last decade as it addresses issues of complexity, scalability, focus and bureaucratic overhead.
But it comes with one huge caveat: Agile hurts secure code development. There, I said it. Someone had to. The Agile process, and even the scrum leadership model, hamstrings development in the area of building secure products. Security is not a freakin’ task card. Logic flaws are not well documented, discreet tasks to be assigned. Project managers (and unfortunately most ScrumMasters) learned security by skimming a ‘For Dummies’ book at Barnes & Noble while waiting for their lattes, but these are the folks making the choices as to what security should make it into the iterations. Just like general IT security, we end up wrapping the Agile process in a security blanket or bolting on security after the code is complete, because the process as we know it is not well suited to secure development.
I know there will be several of you out there who saying “Prove it! Show us a study or research evidence that supports your theory.” I can’t. I don’t have meaningful statistical data to back up my claim. But that does not mean it’s not true, and there is anecdotal evidence to support what I am saying. For example:
- The average Sprint duration of two weeks is simply too short for meaningful security testing. Fuzzing & black box testing are infeasible with nightly builds or pre-release sanity checks.
- Trust assumptions between code modules or system functions where multiple modules process requests cannot be fully exercised and tested within the Agile timeline. White box testing can be effective, but face it – security assessments don’t fit into neat 4-8 hour windows.
- In the same way Agile products deviate from design and architecture specifications, they deviate from systemic analysis of trust and code dependancies. It’s a classic forest through the trees problem: efficiency and focus gained by skipping over big picture details necessarily come at the expense of understanding how the system and data are used as a whole.
- Agile’s great at dividing and conquering what you know, but not so great for dealing with the abstract. Secure code development is not like fixing bugs where you have a stack trace to follow. Secure code development is more about coding principles that lead to better security. In the same way Agile can’t help enforce code ‘style’, it won’t help with secure coding guidelines. (Secure) style verification is an advantage of peer programming and inherent in code reviews, but not intrinsic to Agile.
- The person on the Scrum team with the least knowledge of security, the Product Manager, prioritizes what gets done. Project managers as a general guideline don’t track security testing, and they are not incented to get security right. They are incented to get the software over the finish line. If they track bugs on the product backlog, they probably have a task card buried somewhere, but don’t understand the threats. Security personnel are chickens in the project and do not gate code acceptance they way they traditionally were able to do in waterfall testing, and may have limited exposure to developers.
- The fact that major software development organizations are modifying or wrapping Agile with other frameworks to compensate for security is evidence of the difficulties in applying security practices directly.
The forms of testing that fit within Agile are more likely to get done. If they don’t fit, they are usually skipped (especially at crunch time), or they have to be scheduled outside the development cycle. It’s not just that the granular focus on tasks makes it harder to verify security at the code and system levels. It’s not just that the features are the focus, or that the wrong person is making security decisions. It’s not just that the quick turnaround in code production precludes some forms of testing known to be effective at identifying security issues. It’s not just that it’s hard to bucket security into discreet tasks. It’s all that and more.
We’re not going to see a study that compares Waterfall with Agile for security benefits. Putting together similar development teams to create similar products under two development methodologies to prove this point is not practical. I have run Agile and Waterfall projects of a similar nature in parallel, and while Agile had overwhelming advantages in a number of areas, security was not one of them. If you are moving to Agile, great – but you will need to evolve your Agile process to accomodate security. What do you think? How have you successfully integrated secure coding practices with Agile? This is a FireStarter, so discuss in the comments.
Reader interactions
11 Replies to “FireStarter: Agile Development and Security”
I just thought of something that should have went into my OP.
If Agile (as a set of disciplines) produces code that can be refactored, wouldn’t that be the largest win for application security of all?
Nobody wants to work with legacy code — and worse, security is always the afterthought. A software project may produce the LEAST buggy code the world has ever seen, but if security wasn’t factored in: it could also be the MOST difficult to fix if not properly designed. An application could have an IDEAL performance curve, be scalable, be utilized — but might not be secure.
The best thing about Refactoring is that the DESIGN of the application can be more easily changed. How many security bugs are design issues?
Adrian,
Agile does not replace requirements engineering – especially non-functional requirements like security. Such requirements must be defined and kept in mind from the start of a project – whether the project is Agile or Waterfall does not matter.
When such requirements exists in a clearly written way, the Product Manager is committed to these requirements, when he has to prioritize user stories. But that’s not enough. You need a system testing team that tests your software against these requirements. These tests can follow their own schedule, e.g. if a sprint lasts 2 weeks, security system tests can take every 2nd drop and have a sprint of 4 weeks.
I believe the real problem is the security awareness in your organization. The organization must strive for security awarness not only in the development team, but security must be understood by the Product Manager, Product Marketing and the testing team. Also I see a lot of projects that underestimate the need for a good system test team.
So if you have the boundary conditions with the required awareness in place I believe that Agile does not stand in contrast to secure development. I rather claim that waterfall hinders secure development even more: Why? Because people tend to move inconvenient tasks to the end of a project. However waterfall projects tend to last longer at the end with a high pressure to finally finish the project. In that phase errors happens, but security is nothing that you can quickly implement at the end. Secure development starts with a secure concept from the ground up. It’s hard to secure an almost finished but insecure product.
Therefore I believe that continuous security awareness in contrast with a test team that has a security testing strategy is much more successful.
Dre – thanks for the link!
-Adrian
http://www.microsoft.com/downloads/details.aspx?FamilyID=c4b44860-cfba-494a-ba43-13c4aecf86af&displaylang=en
Great timing, my Black Hat talk this week (http://blackhat.com/html/bh-dc-10/bh-dc-10-briefings.html#Sullivan) covers exactly this topic. If you’re coming to the briefings, stop by for the talk and we’ll get some conversation going.
It’s definitely not impossible to do secure Agile dev. I would say that certain Agile tenets present challenges to secure dev, but you can say the same thing about waterfall. The trick is overcoming those challenges, and I think we’ve done a pretty good job with SDL-Agile. Of course, if you’re using SDL-A and finding shortcomings with it I’d love to know about those too so I can address them.
Adam – Point well taken. Really tried to avoid calling out Agile in ways that applied to all development processes. But Waterfall has evolved to include variations and has been supplemented to account for quality and security shortcomings. Agile needs to do the same, but for different reasons. The link was to specific alterations to Agile to account for security (which is the best approach I have seen to date and still leaves a big gap).
Oh, and I should mention Microsoft uses Secure Development Lifecycle (SDL) and we (Securosis) uses Secure Software Development Lifecycle (SSDL).
-Adrian
What if an Agile process includes security as part of the backlog instead of as the focus of a sprint?
You should know that I’m anti pen-testing in general … and anti vulnerability assessment(s) … but is that a different topic? Also, I’m anti QA … so we’re not able to communicate on the same plane of existence, unfortunately.
In my mind, ALM and SDL are things of the past. The new win is code generated unit (and component+system) testing and templating as part of the platform (e.g. Grails) and fixing the security issues at the platform level by prescribing components that will satisfactorily meet the needs of risk management and application development (e.g. OWASP ESAPI) to a certain risk level (e.g. OWASP ASVS).
What does or doesn’t Agile have to do with any of that? Why bring up assessments and pen-testing in the first place?
Adrian,
I think your final bullet is misleading. If that’s “evidence of the difficulties in applying security practices directly,” then the same argument applies to waterfall, because
Microsoft also wraps waterfall development with SDL processes.
Dre – Not changed my mind at all. Still love Agile. And this is not intended as a ‘straw-man’ argument. It’s been something that has bothered me for a while now. The first few iterations all is great, but you begin to see the self-selecting nature of the QA cycle exclude more complex and systemic tests.
Extreme is test first, Scrum implementations may or may not be. Lots of web shops don’t. Extreme programming is a form of Agile, but Agile with Scrum and XP are not the same. The values are the same, the implementation is not. Modular testing is implied with Scrum because you are supposed to have functional code at the end of the sprint. But the testing is how you define it. Even so, unit tests ad-infinitum are decreasingly useful, and occasionally you really need to have a full regression test, and perform some fuzzing / pen tests. But this is my point … Agile is at the point where it needs to evolve to include additional testing that does not fit within the sprint
-Adrian
Changed your mind on this?
Agile, and it’s subsets including Scrum, all have one benefit to security — they are supposed to ADD application lifecycle management (“hygiene”), not take it away.
Unfortunately, the word Agile has been utilized by some organizations to remove that “hygiene” or to increase velocity.
The whole software industry is operating in a “just ship it” mode or as Guy Kawasaki says, “Don’t worry, be crappy. Revolutionary means you ship and then test”
Agile, including early iterations such as Extreme Programming, never say “ship, then test”. The whole Agile paradigm is test-first development.
Is security-testing included in this? Not yet. Should it be? With proper application lifecycle management, ANY and ALL risk factors are included in the “hygiene” efforts (i.e. mostly process related). Better process is much better than “more assessments” or assessments, period.
Process enhancements will get us to solve these issues that you speak of. Not products or services.
Because Agile is “process enhancement”, it will simply take time to build in what is right for software security. Just because it’s not typically done today doesn’t mean that Agile is the root-cause of any problem. The problem is that we focus too much on products/services (especially around “assessment tools” and “assessment services”) for software security.
You are damn right that this post is a Securosis “FireStarter”… it’s ridiculous that you are making these claims and it sounds like a strawman argument to me…