This is the fourth installment of our Secure Agile Development research. Today’s post discusses one of the toughest parts of bringing security into an Agile program; process modification. The common waterfall development process has cleanly delineated phases, each of which provides an opportunity for security integration, and each security activity must be completed before moving on to the next phase. Agile includes whatever work gets done in the sprint – it does not bend to security so you need to bend security to fit Agile.

Before we get started we want to note that Veracode has asked to license this content when we have finished with the series. As with all our research, our posts and papers are written independently and reflect our what we see at customer sites. That said, we’re very happy when people like what we write enough to want to use it, so we would like to thank Veracode for their support!

The easiest way to grasp the process changes is to contrast a waterfall process against Agile methods. This diagram shows a typical waterfall sequence.

Each waterfall step typically includes one or more security tasks. For example, the design phase includes threat modeling to see which relationships are subject to which types of threat. During development we perform static analysis or regression testing for specific vulnerabilities, Quality Assurance includes fuzzing and exception testing, and release management sets firewall policy and patches the deployment environment. Each of these security tasks must be completed before the next step in the waterfall can begin.

Here is a simple Agile with scrum process diagram. The scrum period is 24 hours, and a sprint is typically 2-4 weeks. At the end of the sprint the code ‘milestone’ is demonstrated, and the next highest priority tasks on the Backlog are assigned to developers.

Following are several recommendations for how to integrate specific security tasks into Agile. Agile cycles are short, and there is no direct mapping from one process to the other. We recommend the common integration points for security controls and testing. We encourage you to think about what works for you, keeping in mind that companies often have multiple development teams, and each may implement Agile differently.

Architecture and Design

The architecture and design phases specify intended behavior: which pieces of code are responsible for which tasks (functions), and the handoffs between different modules (communication and trust). This is where you set security policies, define intended and insecure behaviors, and model threats. Agile has no direct equivalent to waterfall’s architecture and design phases, so you need to determine the best fit based on your team’s skills. Many include threat modeling as a task within user stories, so it is just another development operation for a development team member. Others choose to model threats as the stories are written, by product managers or senior developers, baking the results into design changes that affect task cards. Both methods have pros and cons, but the key is to communicate the desired behavior.

Stories describe how a product should work, so story development is a common place to include security functional requirements. The challenge is more how to communicate requirements than when in the process the communication should take place. It must be simple so developers understand what they are being asked to do. Rather than a long list of requirements try a simple state machine diagram to get the idea across. Clear examples of what you want and what to avoid are best. Simpler is usually better.

Product Management and Task Prioritization

The Product Manager or Product Owner assigns tasks. Their responsibility is to manage incoming features or Stories, break work down into manageable tasks, and prioritize them. They often have the least security knowledge on the team. The issue is less which security tasks to put into the backlog and prioritization phase, and more getting the Product Owner to track security issues and get the tasks onto the queue despite competing priorities.

We recommend working with Product Management to assign tags/labels to security tasks and vulnerabilities so they can be tracked like any other feature or bug. Balance security against other development tasks. Product management has a tendency to starve security tasks off the queue in favor of new features, while security people must resist labeling all their work as critical. If you integrate security into user stories, agree on reasonable priorities for different types of security bugs, and have tools to track security work, your working relationships with product owners will improve.

Development and Debugging

Development teams have not always been tasked with debugging their own code. In the waterfall days Quality Assurance teams were often handed completely untested code. One key area Agile helps address is developers ‘dumping’ code, by requiring verification that code performs its basic function before QA accepts it. Many organizations now require developers to write test cases to verify that submitted code accomplishes its required functions. Some Agile teams take this seriously, giving developers a task card to write tests before a task to code a new feature. Speed and efficiency are key requirements, so these tests become part of the automated build process – which automatically rejects code if it does not pass acceptance tests.

Our recommendations are to make security requirements and tests part of the story, but to select small items that quickly help identify whether code passes or fails to meet requirements. You will not get complete testing in this phase, so don’t bother asking. For new features you want the developer to understand the core security requirement, so have them construct acceptance tests to get an early idea of whether they are on track. For vulnerabilities and flaws ask for security regression tests to be included with the fix to ensure the mistake is not repeated. Stick to one or two simple tests to validate the specific need and leave the rest for later. If your organization automates static testing during the development phase you will need to integrate results into tracking tools so developers can leverage them while the code changes are fresh in their minds.

Quality Assurance and Release Management

Most organizations leave the bulk of testing until Quality Assurances gets their hands on the code to perform complete systems tests. That means penetration testing, dynamic app scans, platform compatibility, and integration testing all happen alongside functional and usability testing. But there is a problem. Did we mention that Agile with scrum does not have a QA phase?

Instead development checks in small pieces of code and demonstrates functionality to the team, a lot more testing occurs earlier in the process. The good news is that you will be moving more security tests than ever before into automated development tests. The bad news is all those other tests don’t simply go away when you go Agile. Internal security requirements or contractual agreements may require you to perform specific tests prior to shipping code, so you need to figure out when and where you can fit them.

Here is an example of what we mean:

Some organizations run different types of Agile processes in parallel, essentially wrapping individual Agile processes into a kind of waterfall hybrid. For example, some organization choose an Agile design process, but run outside the development sprint. Similarly some run quality assurance and release management functions as a waterfall process, but allow development teams to run Agile. You alter the process to suit your needs and the skills of your team members.

Your organization has likely come to grips with these issues already, performing final acceptance testing either as part of the normal sprint processes or as its own process. Our recommendation is to automate as much testing as possible as part of the build process, so issues are closely tied to the most recent changes and developers have an easier time understanding what caused problems.

Your goal is to maintain the Agile pace of development, with security included. That means introducing security in ways that accommodate simple and efficient processes. In many cases this means tradeoffs must be made, but as with all things Agile, try it for a few cycles and make changes and improvements over time. Agile has proven that when you break things down to simple components, you gain new and unique options to address development challenges – the same is true for security. You will find some alternatives that are sufficient to get security into your products, and may find ways to do security testing and integration better than before. We will provide specific examples in our last post of this series. But next we will look at tools and tests.