As we looked back on our research output for the past 2 years it became clear that threat intelligence (TI) has been a topic of interest. We have written no less than 6 papers on this topic, and feel like we have only scratched the surface of how TI can impact your security program.

So why the wide-ranging interest in TI? Because security practitioners have basically been failing to keep pace with adversaries for the past decade. It’s a sad story, but it is reality. Adversaries can (and do) launch new attacks using new techniques, and the broken negative security model of looking for attacks you have seen before consistently misses them. If your organization hasn’t seen the new attacks and updated your controls and monitors to look for the new patterns, you are out of luck.

What if you could see attacks without actually being attacked? What if you could benefit from the experience of higher-profile targets, learn what adversaries are trying against them, and then look for those patterns in your own environment? That would improve your odds of detecting and preventing attacks. It doesn’t put defenders on an even footing with attackers, but it certainly helps.

So what’s the catch? It’s easy to buy data but hard to make proper use of it. Knowing what attacks may be coming at you doesn’t help if your security operations functions cannot detect the patterns, block the attacks, or use the data to investigate possible compromise. Without those capabilities it’s just more useless data, and you already have plenty of that. As we discussed in detail in both Leveraging Threat Intelligence in Security Monitoring and Leveraging Threat Intelligence in Incident Response/Management, TI can only help if your security program evolves to take advantage of intelligence data.

As we wrote in the TI+SM paper:

One of the most compelling uses for threat intelligence is helping to detect attacks earlier. By looking for attack patterns identified via threat intelligence in your security monitoring and analytics processes, you can shorten the window between compromise and detection.

But TI is not just useful for security monitoring and analytics. You can leverage it in almost every aspect of your security program. Our new Applied Threat Intelligence series will briefly revisit how processes need to change (as discussed in those papers) and then focus on how to use threat intelligence to improve your ability to detect, prevent, and investigate attacks. Evolving your processes is great. Impacting your security posture is better. A lot better.

Defining Threat Intelligence

We cannot write about TI without acknowledging that, with a broad enough definition, pretty much any security data qualifies as threat intelligence. New technologies like anti-virus and intrusion detection (yes, that’s sarcasm, folks) have been driven by security research data since they emerged 10-15 years ago. Those DAT files you (still) send all over your network? Yup, that’s TI. The IPS rules and vulnerability scanner updates your products download? That’s all TI too.

Over the past couple years we have seen a number of new kinds of TI sources emerge, including IP reputation, Indicators of Compromise, command and control patterns, etc. There is a lot of data out there, that’s for sure. And that’s great because without this raw material you have nothing but what you see in your own environment.

So let’s throw some stuff against the wall to see what sticks. Here is a starter definition of threat intelligence:

Threat Intelligence is security data that provides the ability to prepare to detect, prevent, or investigate emerging attacks before your organization is attacked.

That definition is intentionally quite broad because we don’t want to exclude interesting security data. Notice the definition doesn’t restrict TI to external data either, although in most cases TI is externally sourced. Organizations with very advanced security programs can do proactive research on potential adversaries and develop proprietary intelligence to identify likely attack vectors and techniques, but most organizations rely on third-party data sources to make internal tools and processes more effective. That’s what leveraging threat intelligence is all about.

Adversary Analysis

So who is most likely to attack? That’s a good start for your threat intelligence process, because the attacks you will see vary greatly based on the attacker’s mission, and their assessment of the easiest and most effective way to compromise your environment.

• Evaluate the mission: You need to start by learning what’s important in your environment, so you can identify interesting targets. They usually break down into a few discrete categories – including intellectual property, protected customer data, and business operations information.
• Profile the adversary: To defend yourself you need to know not only what adversaries are likely to look for, but what kinds of tactics various types of attackers typically use. So figure out which categories of attacker you are likely to face. Types include unsophisticated (using widely available tools), organized crime, competitors, and state-sponsored. Each class has a different range of capabilities.
• Identify likely attack scenarios: Based on the adversary’s probable mission and typical tactics, put your attacker hat on to figure out which path you would most likely take to achieve it. At this point the attack has already taken place (or is still in progress) and you are trying to assess and contain the damage. Hopefully investigating your proposed paths will prove or disprove your hypothesis.

Keep in mind that you don’t need to be exactly right about the scenario. You need to make assumptions about what the attacker has done, and you cannot predict their actions perfectly. The objective is to get a head start on response, narrowing down investigation by focusing on specific devices and attacks. Nor do you need a 200-page dossier on each adversary – instead focus on information needed to understand the attacker and what they are likely to do.

Collecting Data

Next start to gather data which will help you identify/detect the activity of these potential adversaries in your environment. You can get effective threat intelligence from a number of different sources. We divide security monitoring feeds into five high-level categories:

• Compromised devices: This feed provides external notification that a device is suspiciously communicating with known bad sites or participating in botnet-like activities. Services are emerging to mine large volumes of Internet traffic to identify such devices.
• Malware indicators: Malware analysis continues to mature rapidly, getting better and better at understanding exactly what malicious code does to devices. This enables you to define both technical and behavioral indicators to search for within your environment, as Malware Analysis Quant described in gory detail.
• IP reputation: The most common reputation data is based on IP addresses and provides a dynamic list of known bad and/or suspicious addresses. IP reputation has evolved since its introduction, now featuring scores to assess the relative maliciousness of each address. Reputation services may also factor in additional context such as Tor nodes & anonymous proxies, geo-location, and device ID, to further refine reputation.
• Command and Control networks: One specialized type of reputation assessment which is often packaged as a separate feed is intelligence on Command and Control (C&C) networks. These feeds track global C&C traffic to pinpoint malware originators, botnet controllers, and other IP addresses and sites you should watch for as you monitor your environment.
• Phishing messages: Current advanced attacks tend to start with a simple email. Given the ubiquity of email and the ease of adding links to messages, attackers typically find email the path of least resistance to a foothold in your environment. Isolating and analyzing phishing email can yield valuable information about attackers and their tactics.

What Has Changed

As you can see, we have had many of these data sources for years. So why are we talking about TI as a separate function? Because the increasing sophistication of attackers has driven change, which means you need to leverage more and better data to keep pace.

We started seeing more advanced security organizations staffing up their own threat intelligence groups a couple years ago. They are tasked with understanding the organization’s attack surface, and figuring out what’s at risk and most likely to be attacked. These folks basically provide context for which of the countless threats out there actually need to be dealt with; and what needs to be done to prevent, detect, and/or investigate potential attacks. These organizations need data and have been willing to pay for it, which created a new market for security data.

Another large change in the threat intelligence landscape has been the emergence of standards, specifically STIX and TAXXI, which enabled quicker and better integration of TI into security processes. STIX provides a common data format for the interchange of intelligence and TAXXI the mechanism & protocols to send it between originators and consumers of the data. Without these standards organizations needed to do custom integrations with all their active controls and security monitors, which was ponderous and didn’t scale.

So in this case standards have been a very good thing for security.

Addressing the Challenges

You just hit the EZ Button, gather some threat intelligence, and find the attackers in a hot minute, leaving plenty of time for golf. That sounds awesome, right? Okay, maybe it doesn’t work quite like that. Threat intelligence is an emerging capability within security programs. So we (as an industry) need to overcome a few challenges to operationalize this approach:

• Aggregate the data: Where do you collect the intelligence? You already have systems that can and should automatically integrate intelligence, and use it within rules or an analytics engine. The more automation the better so resources can focus preventing attacks or figuring out what happened.
• Analyze the data: How do you know what’s important within the massive quantity of data at your disposal? You need to tune your intelligence feeds and refine rules in your controls and monitors over time. As you leverage intelligence in your security program, you get a feel for what works and what isn’t so useful.
• Actionable data: This takes TI to the next level, with tools automatically updating controls and searching your environment based on threat intelligence feeds. Potentially blocking attacks and/or identify attack indicators before the attacker exfiltrates your data. Existing tools such as firewalls, endpoint security, and SIEM can and should leverage threat intelligence. You will also want your forensics tools to play along, with the ability to leverage external intelligence.
• False positives/false flags: Unfortunately threat intelligence is still more art than science. See if your provider can prioritize or rank alerts. Then you can use the most urgent intelligence earlier and more extensively. Another aspect of threat intelligence to beware is disinformation. Many adversaries shift tactics, borrowing from other adversaries to confuse you. That is another reason not to simply profile an adversary, but to cross-reference with other information to make sure that adversary makes sense in your context.

Now you have a decent idea what we mean by threat intelligence, so in the rest of this series we will focus on how TI can be used effectively in common use cases. These include security monitoring/alerting, incident response/management, and active security controls (both network and endpoint). So stay tuned – we will put this series on the fast track and post most of the research in short order.