This post will discuss why secrets management is needed at all, along with the diverse use cases which teams need it to address. In every case there is some secret data which needs to be sent – hopefully not in plain text – to an application or service. And in every case we want the ability to provide secrets, both when an operator is present and automatically. The biggest single issue is that security around these secrets today is largely absent, and they are kept in cleartext within documents of various types. Let’s dive in.
- API Gateways and Access Keys: Application Programming Interfaces are how software programs interact with other software and services. These API form the basic interface for joint operation. To use an API you must first authenticate yourself – or your code – to the gateway. This is typically done by providing an access key, token, or response to a cryptographic challenge. For ease of automation many developers hard-code access keys, leaving themselves vulnerable to simple file or code inspection. And all too often, even when kept in a private file on the developer’s desktop, keys are accidentally shared or posted, such as to public code repositories. The goal here is to keep access keys secret while still provisioning to valid applications as needed.
- Automated Services: Applications are seldom stand-alone entities. They are typically comprised of many different components, databases, and supporting services. And with current application architectures we launch many instances of an application to ensure scalability and resiliency. As we launch applications, whether in containers or as servers, we must provision them with configuration data, identity certificates, and tokens. How does a newly created virtual machine, container, or application discover its identity and access the resources it needs? How can we uniquely identify a container instance among a sea of clones? In the race to fully automate the environment, organizations have automated so fast that they got out over their skis, with little security and a decided imbalance towards build speed. Developers typically place credentials in configuration files which are conveniently available to applications and servers on startup. We find production credentials shared with quality assurance and developer systems, which are commonly far less secure and not always monitored. They are also frequently shared with other applications and services which should not have access. The goal is to segregate credentials without causing breakage or unacceptable barriers.
- Build Automation: Most build environments are insecure. Developers tend to feel security during development slows them down, so they often bypass security controls in development processes. Build environments are normally under developer control, on development-owned servers, so few outsiders know where they are or how they operate. Nightly build servers has been around for over a decade, with steadily increasing automation to improve agility. As things speed up we remove human oversight. Continuous Integration and Continuous Deployment use automation to speed software delivery. Build servers like Jenkins and Bamboo automatically regenerate application as code, formation templates, and scripts are checked into repositories. When builds are complete we automatically launch new environments to perform functional, regression, and even security tests. When these tests pass, some organizations launch the code in production! Build server security has become an important topic. We no longer have the luxury of leaving passwords, encryption keys, and access tokens sitting in unprotected files or scripts. But just as continuous integration and DevOps improve agility, we need to automate the provisioning of secrets into the process as well, and create an audit trail to prove we are delivering code and services securely.
- Encrypted Data: Providing encryption keys to unlock encrypted volumes and file stores is a common task, both on-premise and for cloud services. In fact, automated infrastructure makes the problem more difficult as the environment is less static, with thousands of services, containers and applications popping in and out of service. Traditionally we have used key management servers designed to handle secure distribution and management of keys, but a number of commercial key management tools (hardware and software) have not been augmented for Infrastructure and Platform as a Service. Additionally, developers demand better API integration for seamless use with applications. This capability is frequently lacking, so some teams use cloud native key management, while others opt for secrets management as a replacement.
- Sharing: Collaboration software has helped development, quality assurance, and product mamagement teams cooperate on projects; even though people in these groups are less and less likely to share office space. User are more likely to work from home, at least part time. In some contexts the issue is how to securely share information across a team of remote developers, but that use case overlaps with having IT share secret data across multiple data centers without exposing it in clear text, or exposed in random files. The databases that hold data for chat and collaboration services tend to not be very secure, and texting certificates to a co-worker is a non-starter. The solution is a central, robust repository, where a select group of users can store and retrieve secrets.
Of course there are plenty more use cases. In interviews we discuss everything from simple passwords to bitcoin wallets. But for this research we need to focus on the issues developers and IT security folks asked about.
Our next post will discuss the core features and functions of a secrets management system, as well as some advanced functions which differentiate commercial options from open source. We want to provide a sense of what is possible, and help guide readers to the subset of functions they need for their use cases.