Secure Agile Development: Building a Security Tool Chain
Now that we have laid out the Agile process it’s time to discuss where different types of security testing fits within it. Your challenge is not just to figure out what testing you need to identify code issues, but also to smoothly fit tests into the framework to help speed testing. You will incorporate multiple testing techniques into the the process, with each tool or technique focused on finding slightly different issues. Developers are clever, so development teams find ways to circumvent security testing if it interferes with efficient coding. And you will need to accept that some tests simply cannot be performed in certain parts of the process, while others can be incorporated in multiple places. To help you evaluate both which tools to consider and how to incorporate them, we offer several recommendations for designing a security “tool chain”. Pre-Sprint Tests and Analysis Threat modeling: Threat modeling is the act of looking for design-level security problems from the perspective of an attacker, and then designing countermeasures. The process enables designers and developers to think about the big picture security of an application or function, then build in defenses, rather than merely focusing on bugs. The classic vectors include unwanted escalation of user credentials, information disclosure, repudiation (i.e.: injecting false data into logs), tampering, spoofing, and denial of service. For each new feature, all these subversion techniques are evaluated against every place a user or code module communicates with another. If issue are identified, the design is augmented to address the problem. In Agile these changes are incorporated into user stories before task cards are doled out. Security defect list: Security defect tracking covers both collecting security defect data and getting the subset of information developers need to address problems. Most organizations funnel all discovered defects into a bug tracking system. These defects may be found in normal testing or through any of the security tests described below. Security testing tools feed defect tracking systems so issues can be tracked, but that does not mean they provide consistent levels of information. Nor do they set the bar for criticality the same. How you integrate and tailor defect feeds from test tools, and normalize those results, is important for effective Agile integration. You need to reach an agreement with the Product Owner on which defects will be addressed in each sprint, and which will be allowed to slide (and for how long). The security defect backlog should be reviewed each sprint. Patching and configuration management: Most software uses a combination of open source and/or commercial code to supplement what the in-house development team builds. For example Apache supports most current web services. Keeping these supplementary components patched is just as necessary as fixing issues in your own code. Agile offers a convenient way to integrate patching and configuration changes at the beginning of each sprint: catalog security patches in supporting commercial and open source platforms, and incorporate the changes into the sprint as tasks. This pre-supposes IT and its production systems are as Agile as development systems, which is regrettably not always the case. Daily Tests Unit testing: Development teams use unit tests to prove that delivered code functions as designed. These tests are typically created during the development process; teams using test-driven or behavior-driven development write them before the code. Unit tests are run after each successful build, and help catch any defects that pop up due to recent changes. Unit tests often include attacks and garbage input to verify that the application is resilient to potential issues outlined during threat modeling. The formal requirement for this type of testing needs to be included in the Agile user stories or tasks – they do not magically appear if you fail to specify them as a requirement. Security regression tests: Regression tests verify that a code change actually fixes a defect. Like unit testing they run after each successful build. Unlike unit test, regression tests each target a known defect, either in the code or a supporting code module. It is common for development teams to break previous security fixes – usually when merging code branches – so security regression tests are a failsafe. With each security tasks to fix a defect, include a simple test to ensure it stays fixed. Manual code inspection: Code reviews, also called peer review, are where a member of the development team examines another developer’s code. They check to ensure that code complies with general standards, but also look for specific implementation flaws such as unfiltered input variables, insufficient user authentication, and unhandled errors. Despite wide adoption of automated testing, about 50% of development shops still leverage manual code review for code quality assessment. Manual efforts may appear inefficient, but manual testing is as Agile as it needs to be. For example, the team chooses whether to perform these tests as part of development, QA, predeployment, or any combination of the above. The task can be assigned to any developer, on any branch of code, and as focused or random as the team decides. We recommend manual testing against critical code modules, supplemented with automated code scanning because manual scanning is repetitive and prone to errors. Manual testing can serve a very valuable security function when properly integrated, by focusing on critical functions (including authentication and encryption), using domain experts to keep an eye on that code and subsequent changes. Every-Sprint (Commit) Tests Static analysis: Static analysis examines the source code of a web application for common vulnerabilities, errors, and omissions within the constructs of the language itself. This provides an automated counterpart to peer review. Among other things these tools generally scan for unhandled error conditions, unfiltered input variables, object availability and/or scoping, and potential buffer overflows. The technique is called “static analysis” because it examines source code rather than execution flow of a running program. Like manual reviews, static analysis is effective at discovering ‘wetware’ problems: issues in code that are directly attributable to programmer error. Better tools integrate well with various
