We will close out this series with a look at several operational considerations for selecting a secrets management platform. There are quite a few secrets management tools, both commercial and otherwise, on the market, and each does things a bit differently. Rather than a giant survey of every product and how it works, we will focus on the facets of these products which enable them to handle the use cases discussed earlier. Central questions include how these platforms deploy, how they provide scalability and resiliency, and how they integrate with the services they supply secrets to? To better distinguish between products you need to understand why they were created, because core functions and deployment models are heavily influenced by a platform’s intended use.

Classes of Products

Secrets management platforms fall into two basic categories: general-purpose and single-purpose. General-purpose solutions provide secrets for multiple use cases, with many types of secrets. General-purpose systems can automatically provision secrets to just about any type of application – from sending user name and password to a web page, to issuing API keys, to dynamic cloud workloads. Single-purpose options – commonly called ‘embedded’ because they install into another platform – are typically focused on one use case. For example the embedded solutions focus on provisioning secrets to Docker containers, and nest into your orchestration manager (e.g.: Swarm, Kubernetes, DC/OS).

This is a critical distinction, because a product embedded into a container manager may not work for non-container use cases. The good news is that many services are deployed this way so they are still useful in many environments, and because these tools leverage existing infrastructure they often integrate well and scale easily. These platforms typically leverage specific constructs of the orchestration manager or container environment to provide secrets. They also tend to make assumptions about how secrets are to be used; for example they may leverage Kubernetes’ ‘namespace’ to enforce policy or the UNIX ‘namespace’ to distribute secrets. Because containers are ephemeral, ephemeral or ‘dynamic’ secrets are often preferred for those secrets managers. The bad news is that some embedded tools assume your cluster is a secure environment, so they can safely pass and store secrets in clear text. Many embedded tools fully encrypt secrets, but they may not support diverse types of secrets or integrate with non-containerized applications.

These specializations are neither good nor bad, depending on what you need for secrets management, but embedded systems may be limited. General-purpose products are typically more flexible and may take more time and to set up, but provide a breadth of functions not generally found in embedded tools.

Deployment Models

Solitary Servers

Common among early tools focused on personal productivity, solitary servers are exactly what the name implies. They typically consist of a central secret storage database and a single server instance that manages it. Basically all functions – including user interfaces, storage management, key management, authentication, and policy management – are handled by a single service. These tools are commonly used via command-line interfaces or API, and work best for a small number of systems.

Client-Server Architecture

The label for this model varies from vendor to vendor. Primary/Secondary, Manager/Worker, Master/Slave, and Service/Agent are just some of the terms to describe the hierarchical relationship between the principal service which manages the repository of secrets, and the client which works with the calling application. This is by far the most common architecture. There is a repository where encrypted secrets are stored, usually a database which is shared or replicated across one or more manager nodes. And each manager can work with one or more agents to support the needs of their service or application.

This architecture helps provide scalability and reliability by spawning new clients and servers as needed. These products often deploy each component as a container, leveraging the same infrastructure as the applications they serve. Many embedded products use this model to scale.

Integration

We already talked about how secrets are shared between a secrets management tool and a recipient, whether human or machine. And we covered integration with container management and orchestration systems, as many tools were designed to do. It’s time to mention the other common integration points and how each works.

• Build Servers: Tools like Jenkins and Bamboo are used by software development teams to automate the building and verification of new code. These tools commonly access one or more repositories to get updated code, grab automation scripts and libraries to set up new environments, connect to virtual or cloud services to run tests, and sign code before moving tested code into another repository or container registry. Each action requires specific credentials before it can take place. Secrets management integration is either performed as a plug-in component to the build server or as an external service it communicates with.
• IT Automation: Automated builds and the power of build managers have vastly improved development productivity, but orchestration tools are what move code at warp speed from developer desktops into production. Chef/Puppet/Ansible are the trio of popular orchestration tools automating IT and development tasks, the backbone of Continuous Integration and Continuous Deployment. Virtually any programable IT operation can be performed with these tools, including most VMware and all cloud services functions offered through API. As with build servers, secrets management typically installs as a component or add-on module of the orchestration tool, or runs as a service.
• Public Cloud Support: Public cloud is a special case. Conceptually, every use case outlined in this series is applicable to cloud services. And because every service in a public cloud is API enabled, it is the ideal playground for secrets management tools. What’s special about cloud services is how integration is managed; most secrets management tools which support the cloud directly integrate with either cloud native identity systems, cloud-native key management, or both. This offers advantages because secrets can then be provisioned in any region, to any supported service within that region, leveraging existing identities. The cloud service can fully define which user can access which secrets. Secrets management can then augment both security and compliance by placing additional usage policies on secrets, or wrapping them in another layer of encryption. There are also cases where customers do not want full integration with their cloud services, preferring to keep certain secrets and encryption keys out of the hands of their cloud service vendor so the vendor cannot be compelled to turn them over by court order.

This concludes our series on Secrets Management. We have received several emails asking questions and clarifying points in previous posts. These comments were mostly addressed in our last post on Features and Functions, and responses have been integrated into that post. We encourage you to leave comments below so the community can see your questions. Comments require moderation on our part, but we usually get to them within 24 hours, so please be patient. We have also been asked to add graphics to describe certain sections, so we will – at least in the research paper, and in this blog series if we can.