Research

One of the hardest things to do in security is to discover what really works. It’s especially hard on the endpoint, given the explosion of malware and the growth of social-engineering driven attack vectors. Organizations like ICSA Labs, av-test.org, and VirusBulletin have been testing anti-malware suites for years, though I don’t think most folks put much stock in those results. Why? Most of the tests yield similar findings, which means all the products are equally good. Or more likely, equally bad. I know I declared the product review dead, but every so often you still see comparative reviews – such as Rob Vamosi’s recent work on endpoint security suites in PCWorld. The rankings of the 13 tested are as follows (in order): Top Picks: Norton Internet Security 2010, Kaspersky Internet Security 2010, AVG Internet Security 9.0 Middle Tier: Avast, BitDefender, McAfee, Panda, PC Tools, Trend Micro, and Webroot Laggards: ESET, F-Secure, and ZoneAlarm The PCWorld test was largely driven by a recent av-test.org study into malware detection. But can one lab produce enough information (especially in a single round of testing) to really understand which product works best? I don’t think so, because my research in this area has shown that 3 testing organizations can produce 10 different results. A case in point is the NSS Labs test from August of last year. Their rankings are as follows, ranked by malware detection rates: Trend Micro, Kaspersky, Norton, McAfee, Norman, F-Secure, AVG, Panda, and ESET. Some similarities, but also a lot of differences. More recently, NSS did an analysis of how well the consumer suites detected the Aurora attacks (PDF), which got so much air play in January. Their results were less than stellar: only McAfee entirely stopped the original attack and a predictable variant two weeks out. ESET and Kaspersky performed admirably as well, but it’s bothersome that most of the products we use to protect our endpoints have awful track records like this. If you look at the av-test ratings and then compare them to the NSS tests, the data shows some inconsistencies – especially with vendors like Trend Micro who are ranked much higher by NSS but close to the bottom by av-test; and AVG which is ranked well by av-test but not by NSS. So what’s the deal here? Your guess is as good as mine. I know the NSS guys and they focus their tests pretty heavily on what they call “social engineering malware,” which are legit downloads with malicious code hidden in the packages. This kind of malware is much harder to detect than your standard malware sample that’s been on the WildList for a month. Reputation and advanced file integrity monitoring capabilities are critical to blocking socially engineered malware, and most folks believe these attacks will continue to proliferate over time. Unfortunately, there isn’t enough detail about the av-test.org tests to really know what they are digging into. But my spidey sense tingles on the objectivity of their findings when you read this report from December by av-test.org and commissioned by Trend. It concerns me that av-test.org had Trend close to the bottom in a number of recent tests, but changed their testing methodology a bit with this test, and shockingly: Trend came out on top. WTF? There is no attempt to reconcile the findings across different sets of av-test.org tests, but I’d guess it has something to do with green stuff changing hands. Moving forward, it would also be great to see some of the application whitelisting products tested alongside the anti-malware suites – for detection, blocking, and usability. That would be interesting. If I’m an end user trying to decide between these products, I’m justifiably confused. Personally, I favor the NSS tests – if only because they provide a lot more transparency on they did their tests. The inconsistent information being published by av-test.org is a huge red flag for me. But ultimately you probably can’t trust any of these tests, so you have a choice to make. Do you care about the test scores or not? If not, then you buy based on what you would have bought on anyway: management and price. It probably makes sense to disqualify the bottom performers in each of the tests, since for whatever reason the testers figured out how to beat them, which isn’t a good sign. In the end you will probably kick the tires yourself, pick a short list (2 or 3 packages) and run them side by side though a gauntlet of malware you’ve found in your organization. Or contract with testing labs to do a test on your specific criteria. But that costs money and takes time, neither of which we have a lot of. The Bottom Line The truth may be out there, but Fox Mulder has a better chance of finding it than you. So we focus on the fundamentals of protecting not just the endpoints, but also the networks, servers, applications, and data. Regardless of the effectiveness of the anti-malware suites, your other defenses should help you both block and detect potential breaches. Share:

Today I read two very different posts on what to look for when hiring, and how to get started in the security field. Each clearly reflects the author’s experiences, and since I get asked both sides of this question a lot, I thought I’d toss my two cents in. First we have Shrdlu’s post over at Layer 8 on Bootstrapping the Next Generation. She discusses the problem of bringing new people into a field that requires a fairly large knowledge base to be effective. Then over at Errata Security, Marisa focuses more on how to get a job through the internship path (with a dollop of self-promotion). As one of our industry’s younger recruits, who successfully built her own internship, she comes from exactly the opposite angle. My advice tends to walk a line slightly in the middle of the two, and varies depending on where in security you want to go. When someone asks me how to get started in security I tend to offer two recommendations: Start with a background as a systems and network administrator… probably starting with the lowly help desk. This is how I got started (yes, I’m thus biased), and I think these experiences build a strong foundation that spans most of the tasks you’ll later deal with. Most importantly, they build experience on how the real world works – even more so than starting as a developer. You are forced to see how systems and applications are really used, learn how users interact with technology, and understand the tradeoffs in keeping things running on a day to day basis. I think even developers should spend some time on the help desk or cleaning up systems – while I was only a mediocre developer from a programming standpoint, I became damn good at understanding user interfaces and workflows from the few years I spent teaching people how to unhide their Start menus and organize those Windows 3.1 folders. Read a crapload of action thriller and spy novels, watch a ton of the same kinds of movies, and express your inner paranoid. This is all about building a security mindset, and it is just as important as any technical skills. It’s easy to say “never assume”, but very hard to put it into practice (and to be prepared for the social consequences). You are building a balanced portfolio of paranoia, cynicism, and skepticism. Go do some police ride-alongs, become an EMT, train in a hard martial art, join the military, or do whatever you need to build some awareness. If you were the kid who liked to break into school or plan your escape routes for when the commies (or yankees) showed up, you’re perfect for the industry. You need to love security. The best security professionals combine their technical skills, a security mindset, and an ability to communicate (Marisa emphasized public speaking skills) with a wrapper of pragmatism and an understanding of how to balance the real world sacrifices inherent to security. These are the kinds of people I look for when hiring (not that I do much of that anymore). I don’t care about a CISSP, but want someone who has worked with users and understands technology from actual experience rather than a library shelf, or a pile of certificates. In terms of entry-level tracks, we are part of a complex profession and thus need to specialize. Even security generalists now need to have at least one deep focus area. I see the general tracks as: Operational Security – The CISO track. Someone responsible for general security in the organization. Usually comes from the systems or network track, although systems integration is another option. Secure Coder – Someone who either programs security software, or is responsible for helping secure general (non-security-specific) code. Needs a programmer’s background, but I’d also suggest some more direct user interaction if they’re used to coding in a closet with pizzas slipped under the door at irregular intervals. Security Assessor (or Pen Tester) – Should ideally come out of the coder or operations track. I know a lot of people are jumping right into pen testing, but the best assessors I know have practical experience on the operational side of IT. That provides much better context for interpreting results and communicating with clients. The vulnerability researcher or penetration tester who speaks in absolutes has probably spent very little time on the defensive or operational side of security. You’ll noticed I skipped a couple options – like the security architect. If you’re a security architect and you didn’t come from a programming or operational background, you likely suck at your job. I also didn’t break out security management – mostly since I hate managers who never worked for a living. To be a manager, start at the bottom and work your way up. In any case, if you’re ready for either of those roles you’re past these beginner’s steps, and if you want to get there, this is how to begin. To wrap this up, when hiring look for someone with experience outside security and mentor them through if they have the right mindset. Yes, this means it’s hard to start directly in security, but I’m okay with that. It only takes a couple years in a foundational role to gain the experience, and if you have a security mindset you’ll be contributing to security no matter your operational role. So if you want to work in security, develop the mindset and jump on every security opportunity that pops up. As either a manager or recruit, also understand the different focus of each career track. Finally, in terms of certifications, focus on the ‘low-level’ technical ones, often from outside security. A CISSP doesn’t teach you a security mindset, and as Shrdlu said it’s insane that something that is supposed to take 5 years of operational experience is a baseline for hiring – and we all know it’s easy to skirt the 5-year rule anyway. I’m sure some of you have

Running old software is bad. Bad like putting a new iPad in a blender. Bad because all software is vulnerable software, and with old software even unsophisticated bad guys have weaponized exploits to compromise the software. So the first of the Endpoint Security Fundamentals technical controls is to make sure you run updated software. Does that mean you need to run the latest version of all your software packages? Can you hear the rejoicing across the four corners of the software ecosystem? Actually, it depends. What you do need to do is make sure your endpoint devices are patched within a reasonable timeframe. Like one minute before the weaponized exploit hits the grey market (or shows up in Metasploit). Assess your (software) assets Hopefully you have some kind of asset management thing, which can tell you what applications run in your environment. If not, your work gets a bit harder because the first step requires you to inventory software. No, it’s not about license enforcement, it’s about risk assessment. You need to figure out the your software vendors’ track records on producing secure code, and then on patching exploits as they are discovered. You can use sites like US-CERT and Secunia, among others, to figure this out. Your anti-malware vendor also has a research site where you can look at recent attacks by application. You probably hate the word prioritize already, but that’s what we need to do (again). Based on the initial analysis, stack rank all your applications and categorize into a few buckets. High Risk: These applications are in use by 50M+ users, thus making them high-value targets for the bad guys. Frequent patches are issued. Think Microsoft stuff – all of it, Adobe Acrobat, Firefox, etc. Medium Risk: Anything else that has a periodic patch cycle and is not high-risk. This should be a big bucket. Low Risk: Those apps which aren’t used by many (security by obscurity) and tend to be pretty mature, meaning they aren’t updated frequently. Before we move on to the updating/patching process, while you assess the software running in your environment, it makes sense to ask whether you really need all that stuff. Even low-risk applications provide attack surface for the bad guys, so eliminating software you just don’t need is a good thing for everyone. Yes, it’s hard to do, but that doesn’t mean we shouldn’t try. Defining the Update/Patch Process Next you need to define what your update and patching process is going to be – and yes, you’ll have three different policies for high, medium and low risk applications. The good news is your friends at Securosis have already documented every step of this process, in gory detail, through our Patch Management Quant research. At a very high level, the cycle is: Monitor for Release/Advisory, Evaluate, Acquire, Prioritize and Schedule, Test and Approve, Create and Test Deployment Package, Deploy, Confirm Deployment, Clean up, and Document/Update Configuration Standards. Within each phase of the cycle, there are multiple steps. Not every step defined in PM Quant will make sense for your organization, so you can pick and choose what’s required. The requirement is to having a defined, documented, and operational process; and to have answered the following questions for each of your categories: Do you update to the latest version of the application? Within how soon after its release? When a patch is released, how soon should it be applied? What level of testing is required before deployment? In a perfect world, everything should be patched immediately and all software should be kept at the latest version. Unless you are talking about Microsoft Vista <grin>. But we all know the world isn’t perfect and there are real economic and resource dependencies to tightening the patch window and buying software updates – and discovering more bugs in the patches themselves. So all these factors need to be weighed when defining the process and policies. There is no right or wrong answer – it’s a matter of balancing economic reality against risk tolerance. Also keep in mind that patching remote and mobile users is a different animal, and you have to factor that into the process. Many of these folks infrequently connect and may not have access to high-bandwidth connections. Specifying a one-day patch window for installing a 400mb patch at a mobile office in the jungle may not be practical. Tools and Automation Lots of tools can help you automate your software updating and patching process. They range from full-fledged asset and configuration management offerings to fairly simple patching products. It’s beyond the scope of this series to really dig into the nuances of configuration/patch management, but we’ll just say here that any organization with more than a couple hundred users needs a tool. This is a topic we’ll cover in detail later this year. The next endpoint control we’ll discuss is Secure Configurations, so stay tuned. Other posts in the Endpoint Security Fundamentals Series Introduction Prioritize: Finding the Leaky Buckets Triage: Fixing the Leaky Buckets Share: