By this point planning should be complete. You have designed your patch and configuration management processes, defined priorities to manage the devices in your environment, figured out which high-level implementation process to start with, discovered the devices in your environment, and performed initial testing to make sure the new technology doesn’t break anything. Now it’s time to integrate the patch and configuration management tools into your environment. Enough of this planning stuff, let’s get down to business! But you won’t actually remediate anything yet – the initial focus is on integrating technical components, installing agents as necessary, and preparing to flip the switch.

Component Overview

We are grouping patch and configuration management together, so we will talk about generic concepts like management servers and agents. A management server might be specifically associated with a patch management product and/or the configuration management environment. Obviously you want leverage between the two, but depending on which technologies you selected you might have different consoles and agents. But the deployment considerations are similar, regardless of the specific use case.

Before we describe specific components we need to briefly go over the inherent security requirements of the different components. If an attacker can change the configuration of a device or apply a malicious patch, it’s pretty much game over. So it’s important to make sure the components are deployed correctly with appropriate security controls. Most solutions use some type of cryptography, both for authentication and to protect communications between components. We are not religious about specific authentication mechanisms (PKI or Windows or whatever), but be sure to check for recent attacks or vulnerabilities for whichever technologies you depend on. You may also want to consider two-factor authentication or some kind of privileged user management technology to better protect the management console.

You will also need to coordinate with the network team to make sure the proper firewall ports are open (and/or proxies identified) to receive updates/new patches from vendors, and to communicate with the relays and/or endpoints using the ports specified by your endpoint security management vendor. Be considerate of the network security team, of course, who will likely resist opening up all sorts of ports throughout the environment. Default deny is still your friend – so when planning the deployment make sure you understand where the servers, distribution points & relays, and agents will be implemented, and how they communicate.

Management Server/Appliance

The management server is the brains of the operation. It holds the policies and provides the focal point for data aggregation, analysis, visualization, and reporting. You have a few options for how to implement the management server, so let’s discuss the pros and cons of each.

• Software: The most common choice is to install software on a dedicated server. Depending on your product this might actually run across multiple physical servers for different internal components such as a back-end database, or to distribute functions for better performance. Some products require different software components running concurrently to manage different functions. This is frequently a legacy of mergers and acquisitions – most products converge on a single software base, although integration may not be as complete as you would expect. Management server overhead is generally fairly low, especially outside large enterprises, so this server often handles some network monitoring, functions as the email MTA (for alerting), and manages endpoint agents. A small-to-medium-sized organization generally only needs to deploy additional servers for load balancing or hot standby. Integration is easy – install the software and position the physical server wherever needed, based on deployment priorities and network configuration, ensuring visibility to the relays and/or agents that need to communicate with it.
• Appliance: In this scenario the endpoint security management software comes preinstalled on dedicated hardware, presumably with a locked-down secure operating system. There is no software to install, so the initial integration is usually a matter of connecting it to the network and setting a few basic options – we will cover the full configuration later. As with a standard server, the appliance usually includes the ability to run multiple functions (though you might need licenses to unlock capabilities).
• Virtual Appliance: The endpoint security management software is preinstalled into a virtual machine for deployment as a virtual server. This is similar to an appliance but requires work to get up and running on your virtualization platform of choice, configure the network, and then set up the initial configuration options as if it were a physical server or appliance.

For now just get the tool up and running so you can integrate the other components. Do not deploy any policies or turn on monitoring yet.

Agents (or Not)

Endpoint agents are, by far, the most varied patch and configuration management components. There are huge differences between the various products on the market, with far more severe performance constraints running on general-purpose workstations and laptops than on dedicated servers. Fortunately, as widely as features and functions vary, the deployment process is consistent.

1. Test, then test more: We know we keep telling you to test your endpoint agents, but that’s not an accident – inadequate testing is the single most common problem people encounter. If you haven’t already, make sure you test your agents on a variety of real-world systems in your environment to make sure performance and compatibility are acceptable. That’s why choosing test devices in the preparation step is so important.
2. Create a deployment package or enable in your EPP tool: The best way to deploy any agent is to use whatever software distribution tool you already use for normal system updates. There is no need to reinvent the wheel here. This means building a deployment package with the agent configured to connect to the patch and/or configuration management server. Remember to account for any network restrictions that could isolate endpoints from the server. In some cases the agent may be integrated into your existing EPP (Endpoint Protection Platform) tool. More often you will need to deploy an additional agent, but if it is fully integrated you can configure and enable it either through the patch/configuration management console or in the EPP tool itself.
3. Activate and confirm agent installation: Once the agent is deployed go back to your patch/configuration management console to validate that systems are covered, agents are running, and they can communicate with the server. Don’t turn on any policies yet – for now just confirm that the agents deployed successfully and are communicating.

Some offerings do not require an agent, per se, but let’s all be clear that software is required on any managed device to check configurations and perform any required remediation. It could be a ‘dissolvable’ agent, which is downloaded as necessary by the endpoint and deleted when it’s no longer needed. Agentless options can work for patch management, because patch scans and patching are both performed on a periodic basis. Configuration management is a bit different – monitoring configuration changes requires fairly frequent assessment, as in couple of hours rather than twice a month – so keep that in mind when determining whether you want to deploy agents. We aren’t hung up on whether or not to use agents – just understand the trade-offs when you choose.

Deployment Models

Making sure the patch and/or configuration management system will scale is critical to integration and deployment. To provide scalability many vendor architectures involve distribution points (also called relays) to aggregate patches, analyze, scan, and normalize and compress aggregated data to be sent back to the central console. Some of the patches to deploy will be large (gigabytes), which means you don’t want every device contacting the mother ship for each download.

So you are likely to have a central server as the main contact point with the vendor’s information service to receive notification of new patches and download packages. Then you will have a bunch of distribution points/relays by location, bandwidth, and numbers of managed devices. Position these relays strategically to make the best use of network bandwidth. They are typically configured as slaves of the central server, receiving policies from it.

You will also want to consider high availability architectures for larger environments. They require configuring management servers and perhaps the relays with hot standbys to handle failover. Pay attention to the server replication mechanism – you shouldn’t lose data if you lose a server. Here are some other considerations when implementing the technology:

• Pay attention to component security, including communications channels and protocols. Hopefully you can piggyback on the existing VPN between your locations, with additional network-layer security between components, including mutual authentication.
• If you plan to allow remote locations to implement their own policies for patch and/or configuration management, now is the time to set up a few test policies and a workflow to verify that your tool can support your requirements. Also make sure to factor in some kind of policy authorization process to ensure each location adheres to the organization’s general policies.

Remote Devices

One of the fun parts of managing thousands of endpoints is that at any time quite a few are not connected to your network. But they still have access to sensitive data and require enforcement of patch and configuration policies. There are two considerations when thinking about these remote devices:

1. Agent Implementation: Lack of full-time network access to these devices raises the challenge of how to get the implemented onto the devices in the first place. So you will want is a process to access every single device and install the agent. We discussed endpoint protection platforms above, and those vendors have solved this problem, so this is another case where piggybacking on that infrastructure can facilitate things; some EPP vendors have open platforms which enable third-party vendors to leverage management functions. Otherwise you might need to remote into the devices or utilize something like a GPO policy to run a script on connection to the network (to pick up email or access a file store, for example).
2. Assessment and Remediation: Once the agent is in place you need to ensure some mechanism establishes a connection every so often, so the device can be inspected and remediated as necessary. The agent should be able to phone home as needed, but it’s your responsibility to define how often and from where. You will have grumpy end users if you download multi-gigabyte patches over satellite connection in the Amazon.

Fortunately software distribution to remote devices is a solved problem, so there is no need to overthink this – just ensure you have suitable policies and infrastructure in place to support remote devices.

Other Integration Points

Your patch and configuration management system will need to integrate with a number of other enterprise systems. The first is the asset repository we mentioned under preparation. It is important to have bi-directional communication with the central asset management environment for both initial discovery and tracking of new devices. If your organization hasn’t deployed any kind of centralized asset management, many patch and configuration management platforms include an integrated capability that can serve as the authoritative source for your shop.

Once a patch and/or configuration change is identified, someone needs to do the work. That may mean tasking the operations team with making a change or installing a patch. Most of those teams live and die by their help desk/trouble ticketing systems, so make sure you can both create new tickets and close the loop when a ticket is marked completed.

Finally, there is tremendous value in sending patch and configuration information to a security monitoring/SIEM system. Configuration changes can indicate malware or other attacks, and when correlated with other enterprise network and server data sources may shorten the detection window for an attack. Likewise, a missing patch on a particular device, combined with information about a specific attack detected by an IPS, can show a clear and present danger to a specific device to trigger action. So ensure your patch/configuration management console can communicate with the SIEM.

Training

The last consideration to mention during integration and deployment is training. Certain staff probably received some training during the testing phase, and now is the time to get everyone else on the operational and security teams educated on how to use the tools, as well as their respective responsibilities and accountabilities. Your responsibility is to make sure that you are prepared to flip the switch, and can take advantage of the new management systems without causing trouble elsewhere.

Once you have the technology integrated and deployed, it’s time to install the policies that govern patch and configuration management. That’s what we will discuss in the next post.