It was simpler back then. You know, back in the olden days of 2003. Viruses were predictable, your AV vendor could provide virus signatures to catch malware, and severe outbreaks like Melissa and SQL*Slammer depended on brittle operating systems and poor patching practices. Those days are long gone, under an onslaught of innovative attacks which leverage professional software development tactics and take advantage of the path of least resistance – generally your employees.

We have written extensively about battling advanced attackers – the top issue facing many security organizations today. From the original Network-based Malware Detection paper, through Evolving Endpoint Malware Detection, and the most recent Early Warning arc: Building an Early Warning System, Network-based Threat Intelligence, and Email-based Threat Intelligence. Finally we took our message to executives with the CISO’s Guide to Advanced Attackers.

But in the world of technology change is constant. Attacks and defenses change, so as much as we try to write timeless research, sometimes our stuff needs a refresh. Detecting advanced malware on the network is a market that has changed very rapidly over the 18 months since we wrote the first paper. Compounding the changes in attack tactics and control effectiveness, the competition for network-based malware protection solutions has dramatically intensified, and every network security vendor either has introduced a network-based malware detection capability or will soon. This makes a confusion situation for security practitioners who mostly need to keep malware out of their networks, and are less interested in vendor sniping and badmouthing.

Accelerating change and increasing confusion usually indicate that it is time to wade in again, to document the changes to ensure you understand the key aspects – in this case, of detecting malware on your network. So we are launching a new series: Network-based Malware Detection 2.0: Assessing Scale, Security, Accuracy, and Blocking, to update our original paper. As with all our blog series we will develop the content independently and objectively, guided by our Totally Transparent Research methodology. But we have bills to pay so we are pleased that Palo Alto Networks will again consider licensing this paper upon completion.

But let’s not pt the cart before the horse – it is time to go back to the beginning, and consider why advanced malware requires new approaches, for both detection and remediation.

Gaining Presence with New Targets
Cloppert’s Kill Chain is alive and well, so the first order of attacker business is to gain a foothold in your environment, by weaponizing and delivering exploits to compromise devices. Following the path of least resistance, it is far more efficient to target your employees and get them to click on a link they shouldn’t. That is not new, but their exploitation targets are. Attackers go after the most widely deployed software, for the greatest number of potential victims and the hest chance of success. This has led them to unpatched operating system vulnerabilities. With recent versions of Windows this exploitation has gotten much harder, which is good thing – for us.

So attackers went after the next most widely distributed software: browsers. Their initial success compromising browsers forced all browser providers to respond aggressively and better lock down their software. Of course we still see edge case problems with older browsers requiring out-of-cycle patches, but browsers have now largely escaped being the path of least resistance. The action/reaction cycle continues, with attackers shifting their attention to other widely used software – particularly Adobe Reader and Java. And once Oracle and Adobe progress there will be a new target. There always is. The only thing we can count on is that attackers will find new ways to compromise devices.

The Role of the Perimeter
Once attackers establish a presence in your network via the first compromised device, they move laterally and systematically toward their target until they achieve their mission. Defensive is the attempt to detect and block malicious software – optimally before it wreaks havoc on your endpoints. Because once malware establishes itself on the device you can no longer rely on endpoint defenses to stop it. We talk to many larger organizations that basically treat every endpoint as a hostile device. If it isn’t already compromised, it will be soon enough. They use preemptive measures, such as extensive network segmentation, to make it harder for attackers to access their targeted data. But what these organizations want is to stop malware from reaching endpoints in the first place.

There is clear precedent for this approach. Years ago anti-spam technology ran on email servers. But blocking technology evolved out to the perimeter, and eventually into the cloud, to shift the flood (and bandwidth cost) of bad email as far away from your real email system as possible. We expect a similar shift in the locus of advanced malware protection, from endpoints to the perimeter. But that begs the question: how can you detect malware on the perimeter? With a network-based malware detection device (NBMD), of course.

As described in the original paper, these devices have emerged to analyze files passing on the wire, and identify questionable files by executing them in a sandbox and observing their behavior. Our next post will revisit that research to delve into how these devices work and how they compliment other controls designed to detect malware elsewhere in your environment.

Insecurity by Obscurity
In the olden days you could just check a file by matching it against a list of signatures from bad files; matches were viruses and blocked. This endpoint-centric blacklist approach worked well … until it didn’t. Today it is largely ineffective – so endpoint protection vendors have shifted focus to a combination of heuristics, cloud-based fuel repositories, IP and file reputation, and a variety of other intelligence-based mechanisms to identify attacks.

But attackers are smart – they have figured out how to defeat blacklists, reputation, and most other current anti-malware defenses. They send out polymorphic files that change randomly – your blacklist is dead. They hijack system files normally exempted from analysis by anti-malware agents. They obscure communications with command and control networks to escape detection by IP reputation defenses. In every way they can, they make it difficult to detect their attacks – defeating our security with their obscurity.

This has created an industry-wide arms race that continues to get fiercer as attackers increase their sophistication. As an example, attackers now add logic to their malware kits to check whether they are executing in a virtual machine – they play dead (or sometimes delay execution for hours or days) in virtual environments, waiting for their chance to run outside the security sandbox. Virtualization is used heavily to make sandboxing practical, so sandbox-aware malware escapes detection by some NBMD devices. New innovative malware techniques make security and accuracy of NBMD devices themselves more important than ever. With the first generation of NBMD technology catching an incremental 40-50% of malware at the perimeter was a win. But that is no longer good enough – we expect much better detection to justify further investment and yet another device on the perimeter.

We see no end in sight for the exponential growth in traffic volume and quantity of malware samples. This imposes a significant scaling requirement on perimeter NBMD equipment – especially because we increasingly expect to deploy NBMD inline for reliable blocking of malicious files. In the face of acute funding and resource shortages, and the cost of investigation and remediation, it has become even more critical to block as much malware at the edge as possible. But going inline changes the latency, security, and reliability requirements of these devices significantly. It is a bad day when an incremental security device knocks down a network or blocks legitimate traffic – as some of you have learned the hard way.

In Network-based Malware Detection 2.0 we will address these changes, and cover the latest and greatest tactics and deployment models to eliminate as much malware on the perimeter of your network as you can. We will resume next week with how these devices detect advanced malware.