We spent the first two posts in this series on the why (Introduction) and how (Detecting Today’s Malware) of detecting malware on the network. But that all assumes the network is the right place to detect malware. As Hollywood types tend to do, let’s divulge the answer at the beginning, in a transparent ploy. Drum roll please… You want to do malware detection everywhere you can. On the endpoints, at the content layer, and also on the network. It’s not an either/or decision. But of course each approach has strengths and weaknesses. Let’s dig into those pros and cons to give you enough information to figure out what mix of these options makes sense for you.

If we recall the last post Detecting Today’s Malware, you have a malware profile of something bad. Now comes the fun part: you actually look for it, and perhaps even block it before it wreaks havoc in your environment. You also need to be sure you aren’t flagging things unnecessarily (the dreaded false positives), so care is required when you decide to actually block something. Let’s weigh the advantages and disadvantages of all the different places we can detect malware, and put together a plan to minimize the impact of malware attacks.

Traditional Endpoint-centric Approaches

If we jump in the time machine and go back to the beginning of Age of Computer Viruses (about 1991?), the main threat vector was ‘sneakernet’: viruses spreading via floppy disks. Then detection on actual endpoint made sense, as that’s where viruses replicated. That started an almost 20-year fiesta (for endpoint protection vendors, anyway), of anti-virus technologies becoming increasingly entrenched on endpoints, evolving three or four steps behind the attacks. After that consistent run, endpoint protection is widely considered ineffective.

Does that mean it’s not worth doing anymore? Of course not, for a couple reasons. First and foremost, most organizations just can’t ditch their endpoint protection because it’s a mandated control in many regulatory hierarchies. Additionally, endpoints are not always connected to your network, so they can’t rely on protection from the mothership. So at minimum you still need some kind of endpoint protection on mobile devices.

Of course network-based controls (just like all other controls) aren’t foolproof, so having another (even mostly ineffective) layer of protection generally doesn’t hurt. And keeping anything up to date on thousands of endpoints is a challenge, and you can’t afford to ignore those complexities. Finally, by the time your endpoint protection takes a crack at detection, the malware has already entered your network, which historically has not ended well. Obviously the earlier (and closer to the perimeter) you can stop malware, the better.

Detecting malware is one thing, but how can you control it on endpoints? You have a couple options:

• Endpoint Protection Suite: Traditional AV (and anti-spyware and anti-everything-else). The reality is that most of these tools already use some kind of advanced heuristics, reputation matching, and cloud assistance to help them detect malware. But tests show these offerings still don’t catch enough, and even if the detection rate is 80% (which it probably isn’t) across your 10,000 endpoints, you would be spending 30-40 hours per day cleaning up infected endpoints.
• Browser Isolation: Running a protected browser logically isolated from the rest of the device basically puts the malware in a jail where it can’t hurt your legitimate applications and data. When malware executes you just reset the browser without impacting the base OS or device. This is more customer-friendly than forcing browsing in a full virtual machine, but can the browser ever be completely isolated? Of course not, but this helps prevent stupid user actions from hurting users (or the organization, or you).
• Application Whitelisting: A very useful option for truly locking down particular devices, application whitelisting implements a positive security model on an endpoint. You specify all the things that can run and block everything else. Malware can’t run because it’s unauthorized, and alerts can be fired if malware-type actions are attempted on the device. For devices which can be subjected to draconian lockdown, AWL makes a difference. But they tend to be a small fraction of your environment, relegating AWL to a niche.

Remember, we aren’t talking about an either/or decision. You’ll use one or more of these options, regardless of what you do on the network for malware detection.

Content Security Gateways

The next layer we saw develop for malware detection was the content security gateway. This happened as LAN-based email was becoming pervasive, when folks realized that sneakernet was horribly inefficient when the bad guys could just send viruses around via email. Ah, the good old days of self-propagating worms. So a set of email (and subsequently web) gateway devices were developed, embedding anti-virus engines to move detection closer to the perimeter.

Many attacks continue to originate as email-based social engineering campaigns, in the form of phishing email – either with the payload attached to the message, more often as a link to a malware site, and sometimes even embedded within the HTML message body. Content security gateways can detect and block the malware at any point during the attack cycle by stopping attached malware, blocking users from navigating toa compromised sites, or inspecting web content coming into the organization and detecting attack code. Many of these gateways also use DLP-like techniques to ensure that sensitive files don’t leave the network via email or web sessions, which is all good.

The weakness of content gateways is similar to the issues with endpoint-based techniques: keeping up with the rapid evolution of malware. Email and web gateways do have a positive impact by stopping the low-hanging fruit of malware (specimens which are easy to detect due to known signatures), by blocking spam to prevent users from clicking something stupid, and by preventing users from navigating to compromised sites. But these devices, along with email and web based cloud services, don’t stand much chance against sophisticated malware, because their detection mechanisms are primarily based on old-school signatures. And once a gateway passes a message through or allows a connection to a web site, the gateway is largely blind. It has no way to detect a compromised device, or to cut off or clean such a device.

Network

Now let’s discuss detecting malware on the network. This generally means at the network perimeter, but some organizations do deploy network security devices internally, and you figure out what architecture is appropriate for your requirements. Network security devices are ubiquitous, so performing some level of detection on them makes sense. Those on the perimeter tend to see and inspect most or all ingress traffic, so malware can be detected at the perimeter before it reaches anything vulnerable. These devices can and should also scrutinize egress traffic, scanning for sensitive data and command and control (C&C) traffic which indicates malware that has successfully eluded the initial defenses.

We spent this entire post talking about methods of detecting malware on the network, so there is no need to rehash that content.

So what’s the catch? Scalability – malware detection can be resource-intensive. This once again raises the almost religious battle about unified threat management (UTM) devices over the past few years. As you may recall, UTM was soundly thrashed in enterprise circles because people refused to believe gateway devices could scale to inspect traffic and do malware analysis at the (near) wire speeds required for network security devices. Three years ago these detractors were not wrong. But the only constant in the security business is change, and a few things have turned the tide here. First is nomenclature. Yesterday’s enterprise UTM device is now called a next-generation firewall. Though the underlying technology of the NGFW differs fundamentally from traditional UTM (for more details check out our Enterprise Firewall paper), both classes of device provide similar capabilities: the ability to enforce application-oriented policies on network traffic.

The second enabler for network security perimeter consolidation is technology evolution. Chips get faster, algorithms improve, and cloud-based resources provide both compute power and much greater scalability for analysis than was previously available on a stand-alone perimeter gateway. So times change, and it’s time to reexamine the ability of network devices to detect malware.

We hate jumping on bandwagons, but we can’t minimize the importance of the innovative technical architectures of NGFW gear hitting the market now. These purpose-built devices process packets differently and enable much cleaner analysis of network traffic, enabling many of these sophisticated functions – including malware detection. Combined with the ability to farm some of the compute overhead and the network effect of shared malware profiles in the cloud, we believe some sort of network-based malware detection will emerge as a key security control over the next 18-24 months. Mostly because we need all the help we can get, and as we discussed regarding endpoint protection, the farther out (the closer to the network perimeter) we can detect and block attacks, the better.

But we understand your (and our) natural skepticism about adding yet another function to the same box. As usual, vendors have provided another device to sit right next to your existing perimeter security gateway and do malware detection. So can you do malware detection on the same device? In good hedging form (there is a US Presidential election coming up, after all), the answer depends on how much traffic needs to be analyzed and how the device leverages cloud services for analysis of malware.

We can’t really wrap up this series on network-based malware detection until we dig much deeper into this: how these gateways leverage the cloud, providing greatly increased scalability for network security devices. That’s the topic of our final post.