Evolving to Security Decision Support: Data to Intelligence
As we kicked off our Evolving to Security Decision Support series, the point we needed to make was the importance of enterprise visibility to the success of your security program. Given all the moving pieces in your environment – including the usage of various clouds (SaaS and IaaS), mobile devices, containers, and eventually IoT devices – it’s increasingly hard to know where all your critical data is and how it’s being used. So enterprise visibility is necessary, but not sufficient. You still need to figure out whether and how you are being attacked, as well as whether and how data and/or apps are being misused. Nobody gets credit just for knowing where you can be attacked. You get credit for stopping attacks and protecting critical data. Ultimately that’s all that matters. The good news is that many organizations already collect extensive security data (thanks, compliance!), so you have a base to work with. It’s really just a matter of turning all that security data into actual intelligence you can use for security decision support. The History of Security Monitoring Let’s start with some historical perspective on how we got here, and why many organizations already perform extensive security data collection. It all started in the early 2000s with deployment of the first SIEM, deployed to make sense of the avalanche of alerts coming from firewalls and intrusion detection gear. You remember those days, right? SIEM evolution was driven by the need to gather logs and generate reports to substantiate controls (thanks again, compliance!). So the SIEM products focused more on storing and gathering data than actually making sense of it. You could generate alerts on things you knew to look for, which typically meant you got pretty good at finding attacks you had already seen. But you were pretty limited in ability to detect attacks you hadn’t seen. SIEM technology continues to evolve, but mostly to add scale and data sources to keep up with the number of devices and amount of activity to be monitored. But that doesn’t really address the fact that many organizations don’t want more alerts – they want better alerts. To provide better alerts, two separate capabilities have come together in an interesting way: Threat Intelligence: SIEM rules were based on looking for what you had seen before, so you were limited in what you could look for. What if you could leverage attacks other companies have seen and look for those attacks, so you could anticipate what’s coming? That’s the driver for external threat intelligence. Security Analytics: The other capability isn’t exactly new – it’s using advanced math to look at the security data you’ve already collected to profile normal behaviors, and then look for stuff that isn’t normal and might be malicious. Call it anomaly detection, machine learning, or whatever – the concept is the same. Gather a bunch of security data, build mathematical profiles of normal activity, then look for activity that isn’t normal. Let’s consider both these capabilities to gain a better understanding how they work, and then we’ll be able to show how powerful integrating them can be for generating better alerts. Threat Intel Identifies What Could Happen Culturally, over the past 20 years, security folks were generally the kids who didn’t play well in the sandbox. Nobody wanted to appear vulnerable, so data breaches and successful attacks were the dirty little secret of security. Sure, they happen, but not to us. Yeah, right. There were occasional high-profile issues (like SQL*Slammer) which couldn’t be swept under the rug, but they hit everyone so weren’t that big a deal. But over the past 5 years a shift has occurred within security circles, borne out of necessity as most such things are. Security practitioners realized no one is perfect, and we can collectively improve our ability to defend ourselves by sharing information about adversary tactics and specific indicators from those attacks. This is something we dubbed “benefiting from the misfortune of others” a few years ago. Everyone benefits because once one of us is attacked, we all learn about that attack and can look for it. So the modern threat intelligence market emerged. In terms of the current state of threat intel, we typically see the following types of data shared within commercial services, industry groups/ISACs, and open source communities: Bad IP Addresses: IP addresses which behave badly, for instance by participating in a botnet or acting as a spam relay, should probably be blocked at your egress filter, because you know no good will come from communicating with that network. You can buy a blacklist of bad IP addresses, probably the lowest-hanging fruit in the threat intel world. Malware Indicators: Next-generation attack signatures can be gathered and shared to look for activity representative of typical attacks. You know these indicate an attack, so being able to look for them within your security monitors helps keep your defenses current. The key value of threat intel is to accelerate the human, as described in our Introduction to Threat Operations research. But what does that even mean? To illustrate a bit, let’s consider retrospective search. This involves being notified of a new attack via a threat intel feed, and using those indicators to mine your existing security data to see if you saw this attack before you knew to look for it: retrospective search. Of course it would be better to detect the attack when it happens, but the ability to go back and search for new indicators in old security data shortens the detection window. Another use of threat intel is to refine your hunting process. This involves having a hunter learn about a specific adversary’s tactics, and then undertake a hunt for that adversary. It’s not like the adversary is going to send out a memo detailing its primary TTPs, so threat intel is the way to figure out what they are likely to do. This makes the hunter much more efficient (“accelerating the human”) by focusing on typical tactics used by likely adversaries. Much of the threat intel available today is focused on data to be pumped into traditional controls, such as SIEM and egress
