As we discussed in Revisiting Security Monitoring, there has been significant change on the security monitoring (SM) side, including the need to analyze far more data sources at a much higher scale than before. One of the emerging data sources is threat intelligence (TI), as detailed in Benefiting from the Misfortune of Others. Now we need to put these two concepts together, to detail the process of integrating threat intelligence into your security monitoring process. This integration can yield far better and more actionable alerts from your security monitoring platform, because the alerts are based on what is actually happening in the wild.

Developing Threat Intelligence

Before you can leverage TI in SM, you need to gather and aggregate the intelligence in a way that can be cleanly integrated into the SM platform. We have already mentioned four different TI sources, so let’s go through them and how to gather information.

1. Compromised Devices: When you talk about actionable information, a clear indication of a compromised device is the most valuable intelligence – a proverbial smoking gun. There are a bunch of ways to conclude that a device is compromised. The first is by monitoring network traffic and looking for clear indicators of command and control traffic originating from the device, such as the frequency and content of DNS requests that might show a domain generating algorithm (DGA) to connect to botnet controllers. Monitoring traffic from the device can also show files or other sensitive data, indicating exfiltration or (via traffic dynamics) a remote access trojan. One approach, which does not require on-premise monitoring, involves penetrating the major bot networks to monitor botnet traffic, in order to identify member devices – another smoking gun.
2. Malware Indicators: As we described in Malware Analysis Quant, you can build a lab and do both static and dynamic analysis of malware samples to identify specific indicators of how the malware compromises devices. This is obviously not for the faint of heart; thorough and useful analysis requires significant investment, resources, and expertise.
3. Reputation: IP reputation data (usually delivered as a list of known bad IP addresses) can trigger alerts, and may even be used to block outbound traffic headed for bad networks. You can also alert and monitor on the reputations of other resources – including URLs, files, domains, and even specific devices. Of course reputation scoring requires a large amount of traffic – a significant chunk of the Internet – to observe useful patterns in emerging attacks.

Given the demands of gathering sufficient information to analyze, and the challenge of detecting and codifying appropriate patterns, most organizations look for a commercial provider to develop and provide this threat intelligence as a feed that can be directly integrated into security monitoring platforms. This enables internal security folks to spend their time figuring out the context of the TI to make alerts and reports more actionable. Internal security folks also need to validate TI on an ongoing basis because it ages quickly. For example C&C nodes typically stay active for hours rather than days, so TI must be similarly fresh to be valuable.

Evolving the Monitoring Process

Now armed with a variety of threat intelligence sources, you need to take a critical look at your security monitoring process to figure out how it needs to change to accommodate these new data sources. First let’s turn back the clock to revisit the early days of SIEM. A traditional SIEM product is driven by a defined ruleset to trigger alerts, but that requires you to know what to look for, before it arrives. Advanced attacks cannot really be profiled ahead of time, so you cannot afford to count on knowing what to look for. Moving forward, you need to think differently about how to monitor.

We continue to recommend identifying normal patterns on your network with a baseline, and then looking for anomalous deviation. To supplement baselines watch for emerging indicators identified by TI.

But don’t minimize the amount of work required to keep everything current. Baselines are constantly changing, and your definition of ‘normal’ needs ongoing revision. Threat intelligence is a dynamic data source by definition. So you need to look for new indicators and network traffic patterns in near real time, for any hope of keeping up with hourly changes of C&C nodes and malware distribution sites. Significant automation is required to ensure your monitoring environment is keeping pace with attackers, and successfully leveraging available resources to detect attacks.

The New Security Monitoring Process Model

At this point it is time to revisit the security monitoring process model developed for our Network Security Operations Quant research. By adding a process for gathering threat intelligence and integrating TI into the monitoring process, you can more effectively handle the rapidly changing attack surface and improve your monitoring results.

Gather Threat Intelligence

The new addition to the process model is gathering threat intelligence. As described above, there are a number of different sources you can (and should) integrate into the monitoring environment. Here are brief descriptions of the steps:

• Profile Adversary: As we covered in the CISO’s Guide to Advanced Attackers, it is critical to understand who is most likely to be attacking you, which enables you to develop a profile of their tactics and methods.
• Gather Samples: The next step in developing threat intelligence is to gather a ton of data that can be analyzed to define the specific indicators that comprise the TI feed (IP addresses, malware indicators, device changes, executables, etc.).
• Analyze Data and Distill Threat Intelligence: Once the data is aggregated you can mine the repository to identify suspicious activity and distill that down into information pertinent to detecting the attack. This involves ongoing validation and testing of the TI to ensure it remains accurate and timely.

Aggregate Security Data

The steps involved in aggregating security data are largely unchanged in the updated model. You still need to enumerate which devices to monitor in your environment, scope the kinds of data you will get from them, and define collection policies and correlation rules. Then you can move on to the active step of collecting the data and storing it in a repository to allow flexible, fast, and efficient analysis and searching.

Security Analytics

The security monitoring process now has two distinct sources to analyze, correlate, and alert – external threat intelligence and internal security data – so some changes to this aspect of the process were necessary.

• Automate TI integration: Given the volume of TI information and the frequency of change, the only way to effectively leverage external TI is to automate ingestion of the data into the security monitoring platform; and automatically updating alerts, reports, and dashboards as appropriate.
• Baseline environment: You don’t really know what kinds of attacks you are looking for yet, so you will want to gather a baseline of normal activity within your environment and then look for anomalies which might indicate compromise and warrant further investigation.
• Analyze Security Data: The analysis process still involves normalizing, correlating, reducing, and tuning the data and rules to generate useful alerts.
• Alert (devices at risk): When a device shows one or more indications that it has been compromised you will trigger an alert to trigger further action.
• Prioritize alerts: Prioritize alerts based on the number, frequency, and types of indicators which triggered them; use these priorities to decide which devices to further inspect, and in what order.
• Deep Collection: Depending on the priority of the alert, you might want to collect more detailed telemetry from the device, and perhaps a network packet capture, to more accurately validate and identify the compromise – as well as to facilitate a forensic investigation if it comes to that.

Action

Once you have an alert and have gathered data relevant to device and attack, you need to determine whether the device was actually compromised or the alert was a false positive. If a device has been compromised you need to escalate the alert, either to an operations team for remediation/clean-up, or to an investigation team for a more formal incident response. To make sure both processes constantly improve you should learn from each validation step, and critically evaluate the intelligence, the policy or rule that triggered the alert, or both.

Our next post will wrap up this series, by going over a Quick Wins scenario that involves implementing this new TI + SM process – including integrating TI data into your monitoring tool and generating/validating some alerts.