As discussed in Application Architecture Disrupted, macro changes including the migration to cloud disrupting the tech stack, application design patterns bringing microservices to the forefront, and DevOps changing dev/release practices dramatically impact building and deploying applications. In this environment, the focus turns to APIs as the fabric that weaves together modern applications. Alas, the increasing importance of APIs also makes them a target.

Historically, enterprises take baby steps to adopt new technologies, experimenting and finding practical boundaries to meet security, reliability, and resilience requirements before fully committing. Requiring a trade-off between security and speed, it may take years to achieve widespread usage of new technologies. But that isn’t fast enough with the expectation that today’s businesses will move fast and break stuff.

As a result, DevOps organizations don’t play by the same rules governing IT adoption of new technologies. In fact, DevOps happened because corporate IT couldn’t move fast enough. These DevOps teams adopt these technologies first and ask for permission later. There needs to be a middle ground where the organization can implement security as part of the tech stack, ensuring adherence to security policies, including protecting critical data, while moving fast enough to deliver in each application sprint.

The Promise of DevSecOps

Getting organizations aligned to deliver secure applications has always been problematic. Incentives and metrics for development teams focus on delivering code on time and within budget. Security can impact those goals by forcing changes and delaying the shipment of new features. Even when security finds an issue and avoids a crippling data breach, it’s tough to be the bearer of bad news. So even when security is right, they are perceived to be wrong.

Doesn’t DevSecOps change all that? The idea is to build security into the development and deployment processes from the start and integrate and automate security testing directly in the pipeline, so security becomes everyone’s business. In this manner, security shifts left (yes, another buzzword) and happens earlier in the development cycle. In effect, DevSecOps makes the entire system more secure, right? That’s the promise anyway.

Now, let’s add another factor to increase the potential impact of DevSecOps, and that’s infrastructure as code (IaC). Everything is code in this world, not just the applications but also APIs, networks, servers, load balancers, etc. These DevSecOps concepts can apply to the entirety of the tech stack. Very exciting indeed!

Yet, the reality is a little different than the promise. DevSecOps requires a genuine cultural shift forcing the traditional walls separating dev, ops, and security to fall. Many a DevSecOps initiative gets scuttled due to politics and organizational resistance to change. Of course, fighting against evolution is not a defendable position in the long term, but short-term it certainly complicates things.

Finally, DevSecOps doesn’t mean security becomes an equal partner. The reality remains security issues are still issues and tend to get lumped together with other features and defects when each application sprint is defined. Security then has to fight to get the changes included in the sprint, which may or may not happen.

How does this relate to API Security, since that’s what we’re talking about, right? It turns out that pretty much every modern development initiative (yes, DevOps) heavily uses APIs. Thus, securely coding and testing the APIs is an integral part of the DevSecOps process. We have to ensure developers both have the proper training and a means to ensure there aren’t issues with the API definitions as the code moves its way through the pipeline.

There’s No Time Like Runtime

Let’s assume that your DevSecOps initiative goes swimmingly. The DevOps teams get it and have instrumented the CI/CD pipeline to ensure API security policies are tested and enforced before any code deployment. But that’s only half the battle. The deployed code is still at risk for manipulation, misuse, and business logic errors that automated tests won’t necessarily catch in the pipeline. What then?

The other half is runtime security, dealing with misuse, drift, human error, or any other issue that violates application (or API) security policies after the code deployment. This requires runtime monitoring to detect potential issues. This API and application security monitoring looks an awful lot like other monitoring techniques. You start by collecting and aggregating data about application/API usage and then watching for signs of misuse. You can (and should) look for clear attack patterns (Indicators of Compromise and Attack), as well as using advanced analytics (machine learning) to see if the application usage varies from a typical usage baseline, potentially indicating malicious intent.

So what happens upon discovering a security issue? Who is responsible for fixing it? Is it Ops? Does the developer have to update the code in the template immediately? Security’s role (or lack thereof) in fixing security issues can cause a lot of frustration amongst security folks, especially when the Ops team doesn’t perceive the same level of urgency to address the issue. As we’ve described, DevOps happened because IT wasn’t responsive enough to the business, so the DevOps team certainly doesn’t want to go back to the old ways of waiting for someone in security to get around to fixing their stuff. Additionally, security will bring a contextual perspective that Dev and Ops will miss because they aren’t immersed in security all day, every day. So it works much better when security and DevOps can work together to address these runtime issues.

Where is the middle ground? It’s a concept that we call guardrails, which are the security policies that the organization cannot violate. We’ve taken to calling them a very technical term – no-no’s – since these are the things that should never happen in a production environment. In the event a guardrail trips, the security team is empowered and expected to fix the issue. Everything else would go into the queue of issues/defects to address in due course by Dev or Ops during a regular sprint.

Defining the no-no’s requires careful consideration since it represents a take action now, ask questions later approach. Relative to API Security, we recommend you start with the OWASP API Top 10 since those are the most common and potentially most damaging issues.

Fool me once…

Whether the issue remediation happens via an automated guardrail or it’s fixed by the Ops team, with the short-term issue averted, you need to think about a more strategic approach to not just handling issues but potentially avoiding them. You know, the whole detection/response cycle versus prevention. How can you make sure you squash these issues as early in the dev process as possible.

Sadly, developers don’t come out of the proverbial womb understanding security and safe coding practices. They need to be taught. We advocate for a security champions program, where developers take on additional responsibility to represent security within their DevOps team. Playing into another critical role security plays in this DevOps and API-centric world, providing guidance is a critical success factor.

Any security issue discovered provides a teaching moment, where developers can get a better sense of how to avoid making the same mistake again. It’s also essential to ensure that you are testing for the security issue within the pipeline, just in case it takes the developer a few times to get it right. What’s important is that the developers learn the lessons of detected security, and the monitors ensure the issues are not missed the next time.

Kumbaya

The key to success in shipping secure code is to ensure that alignment exists within the organization, including a collaborative relationship between security and DevOps. It’s important to gather around the virtual campfire and sing kumbaya every so often and embrace the teams’ mutual dependence. DevOps cannot meet their objective without security, and vice versa. If these teams view themselves as adversaries, as opposed to partners, it cannot work. This seems intuitive and straightforward, but human nature involves finding someone to blame with mistakes are made. It’s critical to make it very clear that everyone is on the same team with aligned objectives. The organization needs these teams to deliver the most functionality possible, on time, within budget, and secure.

So with that, your objective is clear. The development and deployment of modern applications, including a heavy dose of APIs, require a new and different security approach. It’s about more than just shifting left and integrating testing into the pipeline, but also having a clear understanding of the application attack surface and empowering the security team to address the issues that present the most significant risk.