Today we start our latest blog series, which we are calling Evolving Endpoint Malware Detection: Dealing with Advanced and Targeted Attacks – a logical next step from much of the research we have already done around the evolution of malware and emerging controls to deal with it. We started a few years back by documenting Endpoint Security Fundamentals, and more recently looked at network-based approaches to detect malware at the perimeter. Finally we undertook the Herculean task of decomposing the processes involved in confirming an infection, analyzing the malware, and tracking its proliferation with our Malware Analysis Quant research.

Since you were a wee lad in the security field, the importance of layered defense has been drummed into your head. No one control is sufficient. In fact, no set of controls are sufficient to stop the kinds of attacks we see every day. But by stacking as many complimentary controls as you can (without totally screwing up the user experience), you can make it hard enough for the attackers that they go elsewhere, looking for lower hanging fruit. Regardless of how good defense in depth sounds, the reality is that with the advent of increased mobility we need to continue protecting the endpoint, as we generally can’t control the location or network being used. Obviously no one would say our current endpoint protection approaches work particularly well, so it’s time to critically evaluate how to do it better. But that’s jumping ahead a bit. First let’s look at the changing requirements before we vilify existing endpoint security controls.

Control Lost

Sensitive corporate data has never been more accessible. Between PCs and smartphones and cloud-based services (, Jive, Dropbox, etc.) designed to facilitate collaboration, you cannot assume any device – even those you own and control – isn’t accessing critical information. Just think about how your personal work environment has changed over the past couple years. You store data somewhere in the cloud. You access corporate data on all sorts of devices. You connect through a variety of networks, some ‘borrowed’ from friends or local coffee shops.

We once had control of our computing environments, but that’s no longer the case. You can’t assume anything nowadays. The device could be owned by the employee and/or your CFO’s kid could surf anywhere on a corporate laptop. Folks connect through hotel networks and any other public avenues. Obviously this doesn’t mean you should (or can) just give up and stop worrying about controlling your internal networks. But you cannot assume your perimeter defenses, with their fancy egress filtering and content analysis, are in play.

An just in case the lack of control over the infrastructure isn’t unsettling enough, you still need to consider the user factor. You know, the unfortunate tendency of employees to click pretty much anything that looks interesting. Potentially contracting all sorts of bad stuff, bringing it back into your corporate environment, and putting data at risk. Again, we have to fortify the endpoint to the greatest degree possible.

Advancing Adversaries

The attackers aren’t making things any easier. Today’s professional malware writers have gotten ahead of these trends by using advanced malware (remote access trojans [RATs] and other commercial malware techniques) to defeat traditional endpoint defenses. It is well established that traditional file-matching approaches (on both endpoints and mail & web gateways) no longer effectively detect these attacks – due to techniques such as polymorphism, malware droppers, and code obfuscation.

Even better, you cannot expect to see an attack before it hits you. Whether it’s a rapidly morphing malware attack or a targeted attempt, yesterday’s generic sample gathering processes (honeynets, WildList, etc.) don’t help, because these malware files are unique and customized to the target. Vendors use the generic term “zero day” for malware you haven’t seen, but the sad reality is you haven’t seen anything important that’s being launched at you. It’s all new to you.

When we said professional malware writers, we weren’t kidding. The bad guys now take an agile software approach to building their attacks. They have tools to develop and test the effectiveness of their malware, and are even able to determine whether existing malware protection tools will detect their attacks. Even coordinated with reputation systems and other mechanisms for detecting zero-day attacks, today’s solutions are just not effective enough. All this means security practitioners need new tactics for detecting and blocking malware which targets their users.

Evolving Endpoint Malware Detection

The good news is that endpoint security vendors realized their traditional approaches were about as viable as dodo birds a few years back. They have been developing their approaches – the resulting products have reduced footprints, require far less computing resources, and are generally decent at detecting simple attacks. But as we have described, simple attacks aren’t the ones to worry about. So in this series we will investigate how endpoint protection will evolve to better detect and hopefully block the current wave of attacks.

We will start the next post by identifying the behavioral indicators of a malware attack. Like any poker player, every attack includes its own ‘tells’ that enable you to recognize bad stuff happening. Then we will describe and evaluate a number of different techniques to identify these ‘tells’ at different points along the attack chain. Finally we will wrap up with a candid discussion of the trade-offs involved in dealing with this advanced malware. You can stop these attacks, but the cure may be worse than the disease. So we will offer suggestions for how to find that equilibrium point between detection, response, and user impact.

We would like to thank the folks at Trusteer for sponsoring this blog series. As we have mentioned before, you get to enjoy our work for a pretty good price because forward-thinking companies believe in educating the industry in a vendor-neutral and objective fashion.