I have had many conversations over the last few months with firms about to take their first plunge into Agile development methodologies. Each time they ask how to map secure software development processes into an Agile framework. So I picked this Firestarter for today’s retrospective on Agile Development and Security (see the original post with comments).
I am a big fan of the Agile project development methodology, especially Agile with Scrum. I love the granularity and focus it requires. I love that at any given point in time you are working on the most important feature or function. I love the derivative value of communication and subtle peer pressure that Scrum meetings produce. I love that if mistakes are made, you do not go far in the wrong direction – resulting in higher productivity and few total disasters. I think Agile is the biggest advancement in code development in the last decade because it addresses issues of complexity, scalability, focus, and bureaucratic overhead.
But it comes with one huge caveat: Agile hurts secure code development. There, I said it. Someone had to. The Agile process, and even the scrum leadership model, hamstrings development in terms of building secure products. Security is not a freakin’ task card. Logic flaws are not well-documented and discreet tasks to be assigned. Project managers (and unfortunately most ScrumMasters) learned security by skimming a “For Dummies” book at Barnes & Noble while waiting for lattes, but they are the folks making the choices as to what security should make it into iterations. Just like general IT security, we end up wrapping the Agile process in a security blanket or bolting on security after the code is complete, because the process itself is not suited to secure development.
I know several of you will be saying “Prove it! Show us a study or research evidence that supports your theory.” I can’t. I don’t have meaningful statistical data to back up my claim. But that does not mean it isn’t true, and there is ample anecdotal evidence to support what I am saying. For example:
- The average Sprint duration of two weeks is simply too short for meaningful security testing. Fuzzing & black box testing are infeasible in the context of nightly builds or pre-release sanity checks.
- Trust assumptions, between code modules or system functions where multiple modules process requests, cannot be fully exercised and tested within the Agile timeline. White box testing can be effective, but security assessments simply don’t fit into neat 4-8 hour windows.
- In the same way Agile products deviate from design and architecture specifications, they deviate from systemic analysis of trust and code dependencies. It is a classic forest for the trees problem: efficiency and focus gained by skipping over big picture details necessarily come at the expense of understanding how the system and data are used as a whole.
- Agile is great for dividing and conquering what you know, but not so great for dealing with the abstract. Secure code development is not like fixing bugs where you have a stack trace to follow. Secure code development is more about coding principles that lead to better security. In the same way Agile cannot help enforce code ‘style’, it doesn’t help with secure coding guidelines. (Secure) style verification is an advantage of peer programming and inherent to code review, but not intrinsic to Agile.
- The person on the Scrum team with the least knowledge of security, the Product Manager, prioritizes what gets done. Project managers generally do not track security testing, and they are not incented to get security right. They are incented to get the software over the finish line. If they track bugs on the product backlog, they probably have a task card buried somewhere but do not understand the threats. Security personnel are chickens in the project, and do not gate code acceptance the way they traditionally were able in waterfall testing; they may also have limited exposure to developers.
- The fact that major software development organizations are modifying or wrapping Agile with other frameworks to compensate provide security is evidence of the difficulties in applying security practices directly.
The forms of testing that fit Agile development are more likely to get done. If they don’t fit they are typically skipped (especially at crunch time), or they need to be scheduled outside the development cycle. It’s not just that the granular focus on tasks makes it harder to verify security at the code and system levels. It’s not just that features are the focus, or that the wrong person is making security decisions. It’s not just that the quick turnaround in code production precludes effective forms of testing for security. It’s not just that it’s hard to bucket security into discreet tasks. It is all that and more.
We are not about to see a study comparing Waterfall with Agile for security benefits. Putting together similar development teams to create similar products under two development methodologies to prove this point is infeasible. I have run Agile and Waterfall projects of similar natures in parallel, and while Agile had overwhelming advantages in a number of areas, security was not one of them. If you are moving to Agile, great – but you will need to evolve your Agile process to accommodate security.
What do you think? How have you successfully integrated secure coding practices with Agile?
Reader interactions
5 Replies to “Research Revisited: FireStarter: Agile Development and Security”
I have been promoting Secure Development (well – a flavor of an SDL) in one dev department of a big security vendor for some two years. They introduced scrum around five years ago, bot still have some kind of superior phase framework with hard gates imposed on it (I admit).
Nevertheless I think it’s worth the effort applying an SDL to agile teams. For instance, secure coding guidelines can be easily made part of the scrum Done Criteria (code quality submission criteria that are enforced at the end of each sprint). There are other, more or less periodic elements of an SDL like code reviews, reviews of the security architecture, maintenance of the threat model or pen tests that are more difficult to assign, as there are no more hard gates in agile development. Micrsosoft did some great work in developing a “bucket” model as part of their SDL (‘Agile’ section in their ‘SDL Process Guidance’ (http://www.microsoft.com/en-us/download/details.aspx?id=29884) that guarantees to a reasonable degree that these elements don’t get lost, forgotten or ousted by a lack of time. All of their SDL documentation is free, and a lot of their work has pioneered the field.
In conclusion, I think that you can definitely make a difference by introducing an SDL even in agile teams. But by all means you need management backup to be successful. I could see a change in the mind set of the developers. They started to proactively come and ask me about secure design issues way before the next security review. And that’s alraedy half the battle, isn’t it? I didn’t get any metrics that would compare the success of an SDL/waterfall with the success of an SDL/scrum in that company, but I could definitely see progress (I’m no more with that company, but that’s a different story…).
I have been promoting Secure Development (well – a flavor of an SDL) in one dev department of a big security vendor for some two years. They introduced scrum around five years ago, bot still have some kind of superior phase framework with hard gates imposed on it (I admit).
Nevertheless I think it’s worth the effort applying an SDL to agile teams. For instance, secure coding guidelines can be easily made part of the scrum Done Criteria (code quality submission criteria that are enforced at the end of each sprint). There are other, more or less periodic elements of an SDL like code reviews, reviews of the security architecture, maintenance of the threat model or pen tests that are more difficult to assign, as there are no more hard gates in agile development. Micrsosoft did some great work in developing a “bucket” model as part of their SDL (‘Agile’ section in their ‘SDL Process Guidance’ (http://www.microsoft.com/en-us/download/details.aspx?id=29884) that guarantees to a reasonable degree that these elements don’t get lost, forgotten or ousted by a lack of time. All of their SDL documentation is free, and a lot of their work has pioneered the field.
In conclusion, I think that you can definitely make a difference by introducing an SDL even in agile teams. But by all means you need management backup to be successful. I could see a change in the mind set of the developers. They started to proactively come and ask me about secure design issues way before the next security review. And that’s alraedy half the battle, isn’t it? I didn’t get any metrics that would compare the success of an SDL/waterfall with the success of an SDL/scrum in that company, but I could definitely see progress (I’m no more with that company, but that’s a different story…).
@Adrian: good video on Agile vs Security. But why did you have the Flying Spaghetti Monster in there and didn’t even give it credit! 🙂 rAmen
@Chris – Actually I have not. Email me with a link – I’d love to see it! I think you’re right … it takes some creativity and some conscious effort to adapt. There is an OWASP presentation that I did on this a few months after this post if you are interested: http://vimeo.com/15505840
-Adrian
Have you seen the Secure Agile talk I’ve been doing for the past 6 months? If not, check it out. It’s challenging but not impossible — and you have to be willing to adapt. We haven’t figured it all out yet, but who has. Ping me if you want to chat more.