Now that we have covered the concepts behind the Early Warning System, it’s time to put them into practice. We start by integrating a number of disparate technology and information sources as the basis of the system – building the technology platform. We need the EWS to aggregate third-party intelligence feeds and scan for those indicators within your environment to highlight attack conditions. When we consider important capabilities of the EWS, a few major capabilities become apparent:

1. Open: The job of the EWS is to aggregate information, which means it needs to be easy to get information in. Intelligence feeds are typically just data (often XML), which makes integration relatively simple. But also consider how to extract information from other security sources such as SIEM, vulnerability management, identity, endpoint protection, network security, and get it all into the system. Remember that the point is not to build yet another aggregation point – it is to take whatever is important from each of those other sources and leverage it to determine Early Warning Urgency.
2. Scalable: You will use a lot of data for broad Early Warning analysis, so scalability is an important consideration. But computational scalability is likely to be more important – you will be searching and mining the aggregated data intensively, so you need robust indexing.
3. Search: Early warning doesn’t lend itself to absolute answers. Using threat intelligence you evaluate the urgency of an issue and look for the indicators in your environment. So the technology needs to make it easy for you to search all your data sources, and then identify at-risk assets based on the indicators you found.
4. Urgency Scoring: Early Warning is all about making bets on which attackers and attacks and assets are the most important to worry about, so you need a flexible scoring mechanism. As we mentioned earlier, we are fans of quantification and statistical analysis; but for an EWS you nee a way to weight assets, intelligence sources, and attacks – so you can calculate an urgency score. Which might be as simple as red/yellow/green urgency.

Some other capabilities can be useful in the Early Warning process – including traditional security capabilities such as alerting and thresholding. Again, you don’t know quite what you are looking for initially, but once you determine that a specific attack requires active monitoring you will want to set up appropriate alerts within the system. Alternatively, you could take an attack pattern and load it into an existing SIEM or other security analytics solution. Similarly, reporting is important, as you look to evaluate your intelligence feeds and your accuracy in pinpointing urgent attacks. As with more traditional tools customization of alerts, dashboards, and reports enables you to configure the tool to your own requirements.

That brings us to the question of whether you should repurpose existing technology as an Early Warning System. Let’s first take a look at the most obvious candidate: the existing SIEM/Log Management platform. Go back to the key requirements above and you see that integration is perhaps the most important criterion. The good news is that most SIEMs are built to accept data from a variety of different sources. The most significant impediment right now is the relative immaturity of threat intelligence integration. Go into the process with your eyes open, and understand that you will need to handle much of the integration yourself.

The other logical candidate is the vulnerability management platform – especially in light of its evolution toward serving as a more functional asset repository, with granular detail on attack paths and configurations. But VM platforms aren’t there yet – alerting and searching tend to be weaker due to the heritage of the technology. But over time we will see both SIEM and VM systems mature as legitimate security management platforms. In the meantime your VM system will feed the EWS, so make sure you are comfortable getting data out of the VM.

Big Data vs. “A Lot of Data”

While we are talking about the EWS platform, we need to address the elephant in the discussion: Big Data. We see the term “Big Data” used to market everything relating to security management and analytics. Any broad security analysis requires digesting, indexing, and analyzing a lot of security data. In our vernacular, Big Data means analysis via technologies like Hadoop, MapReduce, NoSQL, etc. These technologies are great, and they show tremendous promise for helping to more effectively identify security attacks.

But they may not be the best choices for an Early Warning System. Remember back to the SIEM evolution, when vendors moved to purpose-built datastores and analysis engines because relational databases ran out of steam. But the key to any large security system is what you need to do, and whether the technology can handle it, scalably. The underlying technology isn’t nearly as important as what it enables you do. We know there will be a mountain of data, from all sorts of places in all sorts of formats. So focus on openness, scalability, and customization.

Turning Urgency into Action

Once you get an Early Warning alert you need to figure out whether it requires action, and if so what kind to take. Validation and remediation are beyond our scope here – we have already covered them in Malware Analysis Quant, Evolving Endpoint Malware Detection, Implementing and Managing Patch and Configuration Management, and other papers which examined the different aspects of active defense and remediation. So we will just touch on the high-level concepts.

1. Validate Urgency: The first order of business is to validate the intelligence and determine the actual risk. The early warning alert was triggered by a particular situation, such as a weaponized exploit was in the wild or vulnerable devices. Perhaps a partner network was compromised by a specific attack. In this step you validate the risk and take it from concept to reality by finding exposed devices, or perhaps evidence of attack or successful compromise. In a perfect world you would select an attack scenario and your EWS platform would mine the data and spit out a list of at-risk devices. And at some point the technology will reach this level of automation, but in the meantime you will need to do a bunch of manual searching and mining to identify devices at risk.
2. Remediate: Once you determine an attack is legitimate and demands action, you will decide what action to take. If you haven’t seen the attack yet you can implement a workaround such as additional firewall/IPS rules or a more restrictive endpoint configuration for at-risk devices. Or perhaps you will just add a new SIEM rule to scan for the scenario identified in the warning. You will need to balance being proactive against extra work to defend against an attack that never materializes. Must better to be safe than pwned, but resource realities require you to figure out what to do, and just as importantly what not to do.

Publicizing Quick Wins

Over time, as with any security discipline, you will refine the verification/validation/investigation process. Keep focus on what worked and what didn’t, and tune the process accordingly. It can be a little bumpy when you first deploy an EWS, as you receive a bunch of alerts which lead you down a bunch of dead ends. But the fact is that security is widely regarded as overhead, so you need to focus on getting a Quick Win with any new security technology.

An EWS will find stuff you didn’t know about and help you get ahead of attacks. But that success story won’t tell itself, so when the process succeeds – likely early on – your will need to publicize it, and tell success stories early and often. There are two key areas to focus on. The first is finding proof of an attack in progress which you are able to find and remediate thanks to threat intelligence and the EWS. This illustrates the fact that you will be compromised, so success is a matter of containing the damage and preventing data loss. The value of the EWS is in detecting an attack and shortening the window between exploit and detection.

The second scenario which tells a great story is taking preemptive steps after a business partner is compromised. We have all sat in meetings to discuss the question, “Will that happen to us?” after learning of a breach at a business partner or competitor. If you can definitively say that you get threat intel on emerging attacks (particularly on competitors or partners) and then evaluate whether action needs to be taken, that allays the fear of senior management that Security has no idea what’s happening until it is already over – too late. Even better if you can discuss how preemptive workarounds and remediations, implemented in response to threat intelligence, blocked an actual attack.

You can also tell a story about how threat intelligence helped you evolve security tactics based on what’s happening to other organizations. For instance, if you see what looks like a denial of service (DoS) attack on a set of web servers, but already know (via your intelligence efforts) that DoS is a frequent decoy to obscure exfiltration activities, you have better context to be more sensitive to exfiltration attempts. You would not have this awareness without threat intelligence and a way to apply it to your environment, which clearly justifies the Early Warning System. Finally, to whatever degree you quantify thes time you spend remediating issues and cleaning up compromises, you can show how much time you saved by using external feeds to refine your efforts and prioritize your activities.

So we have now covered all the key topics involved in Building an Early Warning System. Over the next week or so we will assemble the posts into a cohesive whole and package up the paper for licensing and publication. If you disagree with our perspectives here, or have suggestions to make the EWS concept stronger, don’t be bashful. Add a comment to this post and let us know. Your feedback helps make our research better, and we thank you in advance for your help.