Threat intelligence (TI) is hot because it promises to close the gap a bit between attackers and defenders. So we have done considerable research on TI over the past year. We started by talking about the Early Warning System, a monitoring concept that leverages threat intelligence feeds to look for emerging attacks. Then we dove into the kinds of TI you can extract from network traffic, the ability to identify malicious IPs and senders by gathering TI through email, and finally a view of the external world through EcoSystem TI.

As you see there are many different types of threat intelligence feeds, and many ways to apply the technology – both to increase the effectiveness of alerting, and to implement preemptive workarounds based on likely attacks. That is why we call threat intelligence benefiting from the misfortune of others. By understanding attack patterns and other nuggets of information gleaned from studying attacks on other organizations, you can get ahead of the threat.

Okay – you cannot actually get ahead of the threat without a time machine. The threat is already out there, but hopefully it hasn’t been used against you yet. As the networks promote their summer reruns, “If you haven’t seen it, it’s new to you!”

Shortening the Window

We believe one of the most compelling uses for threat intelligence is to help detect attacks earlier in the attack cycle. By looking for attack patterns identified via threat intelligence in your security monitoring/analytics functions, you can shorten the window between compromise and detection.

So we are happy to start a new series called Leveraging Threat Intelligence in Security Monitoring. We will go into depth on how to update your process, in order to integrate your existing malware analysis/threat intelligence gathering function with your security monitoring team’s work. We will be using parts of our Network Security Operations Quant and Malware Analysis Quant process maps to document a new Security Monitoring Process Model leveraging threat intelligence.

We would also like to thank Norse Corp for agreeing to potentially license this content at the end of the process. We build all our public research using our Totally Transparent Research model, so all the research will be posted to the blog first to give everyone an opportunity to provide feedback and comment.

But first things first. We need to set the stage by revisiting the kinds of threat intelligence we have highlighted in our research. This will provide the context you need to understand the kinds of TI feeds you can integrate into your security monitoring environment.

Threat Intelligence Sources

You can get effective threat intelligence from a number of different sources. We can chunk them into major categories to look at for security monitoring:

1. Compromised Devices
2. Malware Indicators
3. Reputation
4. Command and Control Networks

Compromised Devices

The first category of TI is the proverbial smoking gun. Something may look compromised, but until it actually starts acting compromised you may never know. Services are emerging to look for indications on the Internet of devices which either act like bots or communicate with C&C networks. These services are no-touch – you don’t need to install anything on your own network to get a verdict on devices within your network.

How does it work? The intelligence providers penetrate botnets and monitor traffic on C&C networks. Using this information they build lists of (compromised) devices participating in botnets.

Of course these services might detect your own internal honeypots or other malware analysis. So you will want to make sure you have some means of determining which devices should show up on their lists, and which shouldn’t. But being able to identify compromised devices is extremely useful for prioritizing remediation.

Malware Indicators

Malware analysis continues to mature rapidly, getting better and better at understanding exactly what malicious code does. This enables you to define both technical and behavioral indicators to seek out within your environment, as Malware Analysis Quant described in gory detail. Why is this important? The key strategy of classical AV – file blacklisting – is no longer effective, so indicators enable you to detect malware by what it does.

A number of companies offer information on specific malware. You can upload a hash of a malware file – if the recipient has seen it already they match the hash and return their analysis; otherwise you upload the whole file for analysis. The services run malware samples through proprietary sandbox environments and other analysis engines to figure out what they do, build detailed profiles, and provide comprehensive reports which include specific behaviors and indicators. You can search your environment for those indicators to pinpoint possibly compromised devices.

You can also draw conclusions from the kinds of indicators you find. Have those tactics been tied to specific adversaries? Do you see these kinds of activities during reconnaissance, exploitation, or exfiltration? Your analysis can enrich these indicators with additional context for better decisions about the best next step.

Reputation

Since its emergence as a primary data source in the battle against spam, reputation data seems to have become a component of every security control. The most common reputation data is based on IP addresses, and provides a dynamic list of known bad and/or suspicious addresses. This has a variety of uses – learning that a partner’s IP address has been compromised, for instance, should set off alarms, especially if the partner has a direct connection to your network. Traffic to known malware distribution sites, phishing sites, command and control nodes, spam relays, and other sites with bad reputations should be investigated.

Besides IP addresses, pretty much everything within your environment can (and should) have a reputation. Devices, URLs, domains, and files, for starters. If you have traffic going to a known bad site, weird traffic coming from a vulnerable contractor-owned device, or even a known bad file showing up when a salesperson connects to the corporate network, you have something to investigate. If something in your environment develops a bad reputation – perhaps as a spam relay or DoS attacker – you need to know ASAP, hopefully before your entire network gets blacklisted.

C&C Traffic Patterns

One specialized type of reputation which is now often a separate feed is intelligence on command and control (C&C) networks. These feeds track global C&C traffic and use it to pinpoint malware originators, botnet controllers, and other IP addresses and sites your devices should avoid. As mentioned above, these services can also help to identify likely compromised devices within your network that communicate with malware controllers. Integrating this kind of network-based threat intelligence with an egress firewall or web filter might enable you to prevent exfiltration, or enable more aggressive monitoring to identify what attackers are doing.

Of course advanced attackers do not make analyzing C&C traffic easy. They work hard to obscure their communications, including using compromised devices with ‘good’ reputations as C&C nodes to confuse reputation filtering, and frequently changing locations using a variety of sophisticated Domain Generating Algorithms (DGA). Accurately identifying C&C traffic is currently a kind of black magic, but it is a critical aspect of intelligence.

Challenges of Using TI for Security Monitoring

That all sounds cool, right? If you can leverage threat intelligence sources for security monitoring, you should be able to look for patterns impacting other organizations before they impact yours. That’s the concept, anyway – as always there are challenges to making it work in practice.

1. Integration of the data: If you can’t get the TI into your security monitoring environment it doesn’t do much good. The first step is to make sure any threat feeds can be easily integrated. Later in this series we will dig into how.
2. Updating rules/alerts/reports: Once the data is in there you need to actually look for the specified patterns and indicators. That requires a bunch of work on an ongoing basis, to update the security monitoring platform. More realistically you need to automate the process – attacks appear and too rapidly for manual updates to keep up.
3. Triage/Validation: Finally, with TI and updated rules, you will start firing alerts based on new patterns and indicators. But someone still has to validate the attacks and take action. Given the severe security resource and skill constraints in many organizations, unless this process is adequately resourced much of this effort is wasted.

But none of these challenges is insurmountable. It is a question of being aware at the beginning of the process, and factoring the realities into your plans, to avoid issues down the line. Our next post will dig into the Network Security Operations Quant process model to identify places TI can make monitoring functions more effective.