Login  |  Register  |  Contact
Thursday, April 22, 2010

Who DAT McAfee Fail?

By Mike Rothman

There are a lot of grumpy McAfee customers out there today. Yesterday, little Red issued a faulty DAT file update that mistakenly thought svchost.exe was a bad file and blew it away. This, of course, results in all sorts of badness on Windows XP SP3, causing an endless reboot loop and rendering those machines inoperable.

Guess they forgot the primary imperative, do no harm…

To McAfee’s credit, they did own the issue and made numerous apologies. Personally, I think the apology should have come from DeWalt, the CEO, on the blog. But they aren’t making excuses and are working diligently to fix the problem. But that is little consolation for those folks spending the next few days cleaning up machines and implementing the fix.

Yet, there is lots of coverage out there that will explain the issue, how it happened, and how to fix it from LifeHacker or McAfee. You’ll also get some perspective on how this provided an opportunity to test those incident response chops. What I want to talk about is understanding the risk profile of anti-malware updates, and whether & how your internal processes should change in the face of this problem.

First off, no one is immune to this type of catastrophic failure. It happened to be McAfee this time, but anti-malware products work at the lowest layers of the operating system, and a faulty update can really screw things up. Yes, the AV vendors have mature QA processes, which is why you don’t see this stuff happening much at all. But it can, and likely will again at some point.

Yes, you could decide to ditch McAfee, although I’d imagine they’ll be retooling their QA processes to ensure this type of problem doesn’t recur. But that’s a short-term emotional reaction. The real question revolves around how to deal with anti-malware updates. It’s always been about balancing the speed of detection with the risk of unintended consequences (breaking something). So you basically have three choices for how to deal with anti-malware updates:

  1. Automatic updates – This represents the common status quo. The AV vendor issues a release, you get it and install it with no testing or any other mechanisms on your end. To be clear, a vast majority of end users are in this bucket.
  2. Test first – You can take the update and run it through a battery of tests to see if there is a problem before you deploy. This option is pretty resource intensive, because you tend to get multiple updates per day from the vendor; it also extends the window of vulnerability by the length of your testing and acceptance pipeline.
  3. Wait and listen – The last approach is basically to wait a day or two day before installing updates. You peruse the message boards and other sources to see if there are any known issues. If not, you install. This also extends the window of exposure, but would have avoided the McAfee issue.

There is no right answer. Most organizations opt for the quickest protection possible, which means automatic updates to minimize the window of vulnerability. But it gets back to your organization’s threshold for risk. I don’t think the “test first” option is really viable for an organization. There are too many updates. I do think “wait and listen” can make sense for the vast majority of companies out there.

But how does wait and listen work against a zero-day attack? In this case it still works okay, because you can always do a manual test or take the risk of sending out an update before the waiting period is over. And in reality, the signature updates for a 0-day usually take 8-18 hours anyway. But there is a risk you might get nailed in the time between an update arrives and when you deploy it. In that case, hopefully you’ve managed expectations with the senior team regarding this scenario.

I’d be remiss if I didn’t at least mention the need for layers beyond anti-malware. Especially when deciding whether to install an AV update. There are alternative mitigations (at the perimeter or on the network, for example) for most 0-day attacks, which could lessen the impact and spread of an attack. Those can often be made immediately, and are easier to reverse than an install that touches every desktop.

So it’s unfortunate for McAfee and they’ll be cleaning up the mess (in market perception and customer frustration) for a while. And as I told the AP yesterday, fortunately this kind of issue is very rare. But when these things do happen, it’s a train wreck.

—Mike Rothman

Database Security Fundamentals: Auditing Events

By Adrian Lane

I realized from my last post that I made a mistake. In my previous post on Auditing Transactions, attempting to simplify database auditing, I instead made it more complicated. What I want to do is to differentiate between database auditing through the native database transactional audit trail, from other forms of logging and event collection. The reason is that the native database audit trail provides a sequence of associated events, and whether and when those events were committed to disk. Simple events do not provide the same degree of context and are not as capable of providing database state. If you need application context and state – perhaps for Sarbanes-Oxley – you need the audit log. Make no mistake: there are simpler and less invasive ways of collecting data. They also provide an alternative – and in some cases clearer – picture of events. For example, it’s a heck of a lot easier to get data from syslog that native audit. And if all you are interested in is when patches are installed, syslog is a better source of information. If you are only interested in failed login attempts, a login trigger is far more efficient.

The entire purpose of this Database Security Fundamentals series is to create a set of steps, which can each be performed in about an afternoon’s time, to secure your database. I believe the entire sequence can be completed in a week. My goal is to provide clarity and simplicity for database and IT administrators who do not have time to learn and deploy advanced security measures, and are instead interested in raising the security bar without spending weeks or months on the project.

So I want to step back and clarify that the last post is specifically for at those who must use native database audit, primarily to populate reports or fulfill regulatory controls, with security as a secondary goal. And yes, compliance of some sort has become a fundamental requirement for the majority of DBAs. For the rest of you, we’ll dig into simple event collection for security events. If you are interested in a few simple events, but not enough to justify the burden of audit, this phase will be more useful to you.

  1. Define events: The goal here is to figure out what you need, or what others want from you. Installation of patches, alteration of specific permissions settings, granting of public roles, insertion of stored procedures, ad-hoc database access, use of management tools like Toad, adding views, 3 or more failed login attempts, and just about anything involving DBA capabilities are common concerns. These are all simple events with frequency rates low enough not to overwhelm you.
  2. Determine collection methods: Based on which events you want, select a data collection method or two that gather the data you need. There are a lot of ways to gather event data. System tables, command line tools, triggers, syslog, trace options, etc.
  3. Write scripts: To make this easier on yourself script your queries, or turn them into stored procedures, or both. Create the scripts to collect the events and, if needed, filter out what you don’t need. Use whatever scripting language you are comfortable with. Keep in mind that it is often useful to have the scripts make follow-up queries to reference other data sources, and being able to recursively gather additional information based upon simple if-then or where comparisons on data will save you a lot of work. User permission mapping is one such example, as the groups and roles a user belongs to could be a complex set of queries, depending on which platform you are using. You may want to send yourself an email for more critical events that need urgent attention.
  4. Implement: Deploy your scripts and test. Annoying though it may be, you will want to set up a specific user account with just enough privileges to perform the data collection. Secure these scripts so unprivileged users cannot use or modify them. You will want to set up a secure place to dump the results, and if necessary archive and remove files so they don’t take up too much disk space.
  5. Set review schedule: The data you collect is only valuable if you use it, so get in the habit of reviewing the results for anomalies. If security is your goal, plan on spending a few minutes every day on this, and setting alerts on the one or two events that absolutely, positively, look suspicious.
  6. Archive the scripts and document: Keep a copy of the scripts and notes on what you implemented for future reference.

For a single database I find that I can create and test the scripts in an afternoon. Another few hours to set up the user accounts, cron jobs, or archive scripts. After that the entire process is pretty much self-sustaining as long as you stay on top of event review. Some of you who are the lone DBA at your job will consider this step in the Fundamentals series silly. I have had DBAs ask me, “Why would you set up a script to track your own work? Why would I send myself a reminder that I just added a table view?” Remember that this is meant to catch stuff that should not be happening, or events you were not aware of, like someone else in IT making changes to be ‘helpful’. Or when an attacker tries to compromise a database. This afternoon’s effort will all seem worth it when you have your first ‘WTF?’ moment a few months from now, when some web programmer changes the database without telling you.

More advanced methods

I intended to leave database activity monitoring out of this discussion. Monitoring is an advanced database security option, and does not fit into this simpler Essentials series. But those tools provide far more advanced data collection and storage capabilities, policies, and reporting. If the number of events to collect, or of databases grows, or if the policies and reports you need grow beyond a handful, you will need to look into database activity monitoring platforms to automate the work. But that effort will require serious investigation and investment, and will take a lot longer.

—Adrian Lane

Whitepaper Released: Quick Wins with Data Loss Prevention

By Rich

Two of the most common criticisms of Data Loss Prevention (DLP) that comes up in user discussions are a) its complexity and b) the fear of false positives. Security professionals worry that DLP is an expensive widget that will fail to deliver the expected value – turning into yet another black hole of productivity. But when used properly DLP provides rapid assessment and identification of data security issues not available with any other technology.

We don’t mean to play down the real complexities you might encounter as you roll out a complete data protection program. Business use of information is itself complicated, and no tool designed to protect that data can simplify or mask the underlying business processes. But there are steps you can take to obtain significant immediate value and security gains without blowing your productivity or wasting important resources.

In this paper we highlight the lowest hanging fruit for DLP, refined in conversations with hundreds of DLP users. These aren’t meant to incorporate the entire DLP process, but to show you how to get real and immediate wins before you move on to more complex policies and use cases.

I like this paper, and not just because I wrote it. Short, to the point, with advice on deriving immediate value as opposed to kicking off some costly and complex process. This paper is the culmination of the Quick Wins in DLP blog series I posted, all compiled together with a pretty picture or two.

Special thanks to McAfee for licensing the report.

You can download the paper directly, or visit the landing page, where you can leave comments or criticism, and track revisions.


Wednesday, April 21, 2010

Incite 4/21/2010: Picky Picky

By Mike Rothman

My kids are picky eaters. Two out of the three anyway. XX1 (oldest daughter) doesn’t like pizza or hamburgers. How do you not like pizza or hamburgers? Anyway, she let us know over the weekend her favorite foods are cake frosting and butter. Awesome.

XY (boy) is even worse. He does like pretty much all fruits and carrots, but will only eat cheese sticks, yogurt and some kinds of chicken nuggets – mostly the Purdue brand. Over the weekend, the Boss and I decided we’d had enough.

At least the boy will be able to eat in the future... Basically he asked for lunch at the cafe in our fitness center and said he’d try the nuggets. They are baked and relatively healthy (for nuggets anyway). The Boss warned him that if he didn’t eat them there would be trouble. But he really wanted the chips that came with the nuggets, so he agreed.

And, of course, decided he wasn’t going to eat the nuggets. And trouble did find him. We basically dictated that he would eat nothing else until he finished two out of the three nuggets. But he’s heard this story before and he’d usually just wait us out. And to date, that was always a good decision because eventually we’d fold like a house of cards. What kind of parents would we be if we didn’t feed the kid?

So we took the boy to his t-ball game, and I wouldn’t let him have the mini-Oreos and juice bag they give as snacks after the game. He mentioned he was hungry on the way home. “Fantastic,” I said. “I’ll be happy to warm your nuggets when we get home.” Amazingly enough, he wasn’t hungry anymore when we got home. So he went on his merry way, and played outside.

It was a war of attrition. He is a worthy adversary. But we were digging in. If I had to lay odds, it’s 50-50 best case. The boy just doesn’t care about food. He must be an alien or something.

At dinnertime, he came in and said he was really really hungry and would eat the nuggets. Jodi dutifully warmed them up and he dug in. Of course, it takes him 20 minutes to eat two nuggets, and he consumed most of a bottle of ketchup in the process. But he ate the two nuggets and some carrots and was able to enjoy his mini-Oreos for dessert.

The Boss and I did a high five, knowing that we had stood firm and won the battle. But the war is far from over. That much I know.

– Mike

Photo credits: “The biggest chicken nugget in the known universe” originally uploaded by Stefan

Incite 4 U

  1. From fear, to awareness, to measurement… – Last week I talked about the fact that I don’t have enough time to think. Big thoughts drive discussion, which drives new thinking, which helps push things forward. Thankfully we security folks have Dan Geer to think and present cogent, very big thoughts, and spur discussion. Dan’s latest appeared in the Harvard National Security Journal and tackles how the national policy on cyber-security is challenged by definition. But Dan is constructive as he dismantles the underlying structure of how security policies get made in the public sector and why it’s critical for nations and industries on a global basis to share information – something we are crappy at. Bejtlich posted his perspectives on Dan’s work as well. But I’d be remiss if I didn’t at least lift Dan’s conclusion verbatim – it’s one of the best pieces of writing I’ve seen in a long long time… “For me, I will take freedom over security and I will take security over convenience, and I will do so because I know that a world without failure is a world without freedom. A world without the possibility of sin is a world without the possibility of righteousness. A world without the possibility of crime is a world where you cannot prove you are not a criminal. A technology that can give you everything you want is a technology that can take away everything that you have. At some point, in the near future, one of us security geeks will have to say that there comes a point at which safety is not safe.” Amen, Dan. – MR

  2. Phexting? – Researchers over at the Intrepidus Group published a new vulnerability for Palm WebOS devices (the Pre) that works over SMS (text messaging). These are the kinds of vulnerabilities that keep me up at night since I started using smart phones. As with Charlie Miller’s iPhone exploit from last year, sending a malicious text message could trigger actions on the phone. Charlie’s attack was actually more complex (and concerning) since it operated at a lower level, but none of these sound fun. For those of you who don’t know, an SMS is limited to 160 characters of text, but modern phones use that to support more complex actions – like photo and video messages. Those work by specially encoding the SMS message with the address of the photo or video that the phone then automatically downloads. SMS messages are also used to trigger a variety of other actions on phones without user interaction, which opens up room for manipulation and exploit… all without anything for you to notice, except maybe the radiation burns in your pocket. – RM

  3. Time to open source Gaia – With additional details coming out regarding the social engineering/hack on Google, we are being told that the source code to the Gaia SSO module was a target, and social engineering on Gaia team members had been ongoing for two years. While attackers may not have succeeded in inserting a Trojan, easter egg, or other backdoor in the source code, the thieves will certainly perform a very thorough review looking for exploitable defects. If I ran Google I would open up the source code to the public and ask for help reviewing it for defects. I can’t help laughing at the thought, but it would be a fun Summer of Code 2010 project. That would help Google, and the code could help developers of Google Apps. Otherwise Google has to pray that their coders and testers are better than the hackers. At the very least they had better conduct their own internal review and create some monitoring policies around any discovered defects. That way they can detect outsiders attempting to exploit the service and maybe, just maybe, trace the attacks back to the source. It also might not be a bad thing to engage law enforcement early in the process. – AL

  4. Botnet detection is not a market… – Sometimes we need to take a step back and remember cause and effect. We are seeing a lot of technology focused on botnet detection. Two companies from my hometown, Pramana and Damballa, are productizing technology to detect and presumably block bot activity. Damballa works on the network, and Pramana on web sites themselves. I know of another company launching a similar network-based technology (though broader – just ask them) in a few weeks as well. Pramana is taking on CAPTCHA, but that will be hard because it’s free and already works well enough. Regarding network-based activity, we have to remember the bot is the effect, not the root cause, and detecting and blocking bot traffic is just treating a symptom. Which is not to say that understanding what devices are compromised isn’t important, but there are plenty of ways to do that, and ultimately this so-called network bot detection market needs to be part of the perimeter boxes, rather than a stand-alone offering. – MR

  5. Not THAT kind of revolution – Are the Israelis making fun of Apple? Is their treatment of “the revolutionary iPad” as a radical and subversive threat a joke? When word popped up that Israel has banned Apple’s iPad from entering the country, and is confiscating them at the airport, I figured something pretty bad having to do with security was going on. Maybe Apple had botched the WPA2 implementation in such a way that passwords were being leaked. And based on Israeli’s posture in the media, they seemed serious so I didn’t think this was an idle threat. But I don’t see anything different about the networking capabilities between the iPad and the iPhone, with the latter already prevalent in Israel. And if the iPhone has not damaged their networks, the iPad certainly is not going to do so. So what’s up? There have been many reported instances of DHCP anomalies, but worst case it’s denial-of-service lite. It’s not like Hezbollah is going to be buying a bunch of iPads and terrorizing Tel Aviv by disrupting Internet service at coffee shops. So this is either financially or politically motivated, or both. – AL

  6. WIPS it good… – Yes, now I have that Devo anthem ringing in my head, but the question still remains whether WIPS is something folks need, or is that capability already subsumed into deployed wireless access switches or branch office boxes? The folks at Accuvant are of the opinion that WIPS is important, and hopefully not just because they sell and deploy WIPS gear. Their contention is that WIPS provides both security and visibility into wireless network performance, and adds real value. Hmmm. What other market was initially about security and then evolved to be more about network operations? Right, network behavioral analysis, and we all know that didn’t work out too well. So why is WIPS different? I don’t think it is, though Accuvant does point out that you can either build a WIPS overlay or use existing switches to deploy the technology. If you get it basically for free, more visibility is better than less – which is pretty much the definition of a feature, not a stand-alone solution. – MR

  7. Attack of the cloud – According to the VoIPTechChat blog a variety of SIP brute force attacks are originating from within Amazon EC2. The attacks appear to have spun up some virtual machines in Amazon’s cloud, then used them to attack the outside world. This is interesting on a couple levels, and I highly recommend you read the original post with Amazon’s responses. This is a pretty cool concept that Chris Hoff has talked about – the bad guys spin up some EC2 instances, use them for nefarious purposes, then shut them down. Better yet – they can do all this with stolen credit card numbers, route it through proxies, and be damn hard to catch. – RM

  8. Does HPCom get close to the TippingPoint? – HP has closed its acquisition of 3Com, and now TippingPoint is an HP property to be folded (along with the rest of 3Com) into the ProCurve division. So the real question for security folks is what happens to TippingPoint? Historically, HP has paid lip service (at best) to security. The ProCurve guys have done some work there, including a security blade for their switches and a NAC OEM from StillSecure, but they really haven’t played into the enterprise. Which on the surface makes TippingPoint look like a square peg. Obviously folks who already have a big commitment to TP should be pushing their HP reps for answers and probably deferring big deployments until the strategy is clarified. For those looking at TP, again fly the flag of caution, because in a space as competitive as network security, nothing less than a commitment from HP to enterprise security will keep TP competitive. – MR

—Mike Rothman

Tuesday, April 20, 2010

Google: An Example of Why Single Sign on Sucks

By Rich

According to the New York Times, when Google was hacked during the recent China incident, their single sign on system was specifically targeted. The attackers may have accessed the source code, which gives them some good intel to look for other vulnerabilities. There’s speculation they could have also added a back door to the source code, but I suspect that even if they did this, given how quickly Google detected the intrusion, any back doors probably didn’t make it into backups and might be easy for Google to spot and remove.

I’ve never been a fan of single sign on (SSO). Its only purpose is to make life easier for the users at the expense of security. All you need to do is compromise one password and you get access to everything. It’s okay if you use strong authentication (like tokens), but crap if you run it solo.

Not that we can expect all our users to remember 25 complex passwords. That’s why passwords alone as an authentication mechanism also suck.

If you can’t roll out strong authentication, I tend to recommend reduced sign on – instead of one password you have the user remember somewhere between 3-5 to compartmentalize. Drop the 90-day rotations, because they only make life harder without actually improving security, and encourage passphrases rather than the silly 10-character, must-have-a-number, non-alpha-character, and 3-Wingdats-symbols-drawn-in-crayon crap.

Personally I use a password vault (1Password. Technically it’s close to SSO in that if someone gets the password to my vault I’m in deep trouble, but to do that they would need to take over my personal system, and it’s pretty much game over at that point anyway. I don’t have to worry if someone compromises a web forum that they’ll use my password there to access my bank account, since they all use different credentials, and I don’t even know what they all are.

Update: Two points I forgot:

  1. I don’t do much with Google, but I do have different accounts set up for when I need to compartmentalize services.
  2. My bank passwords are not in 1Password – those I keep memorized, because I’m a paranoid freak.


Monday, April 19, 2010

Level 4 Apathy

By Mike Rothman

I was perusing some of my saved links from the past few weeks and came across Shimmy’s dispatch from the ETA (Electronic Transaction Association) show, which is a big conference for payment processors. As Alan summarized, here are the key takeaways from the processors:

  • They view the PCI Council as not caring about Level 3 and 4 merchants. Basically a shark with no teeth.
  • They don’t see smaller merchants as a big risk.
  • They believe their responsibility ends when a ‘program’ is in place.

Alan uses the rest of his post to beat on the PCI scanning shylocks, who are offering services for $1 per merchant, to get their vulnerability scan checkbox and to fill out the SAQ.

But my perspective is a bit different. Right there, in the flesh, is the compliance-centric mindset. It’s not about outcomes, it’s about checking the box. And we can decide to get all upset about it, but that would be a waste of time. You see, apathy is usually a result of some kind of analysis (either conscious or unconscious). I suspect the processors have done the math and decided to focus their risk management on the places where they lose the most money – presumably the Level 1 and 2 merchants.

Now I haven’t seen the fraud reports from any of these folks, but I presume they do a bit of analysis on to where their ‘shrinkage’ occurs, and if a large portion of it was Level 3 and 4 merchants, then Mr. Market would expect them to be much more aggressive about making real security changes at that level. But they aren’t, so the only conclusion I can draw is that even though (as Alan says) 85% of the incidents take place at smaller merchants, it’s probably only a small portion of the total dollars in fraud. To be clear, I could be making that up, and/or the processors could just be crappy at understanding their risk profiles. But I don’t think so.

I think as an industry we really have to start thinking about the point of diminishing returns. Where is the line where increasing our efforts to secure small companies just doesn’t matter? You know, where the economic benefit of reduced fraud is outweighed by the cost of making those security improvements. Seems like the PCI Council is already there. Of course, the trade press will still get all aflutter about the builder or shop owner whose accounts are looted for $100K or $500K, and then they go out of business.

That’s sad, but it seems the card value chain is focused on stopping the $100M losses, and is willing to accept the $100K fraud. Predictably, the system is figuring out how to game the lower levels of the regulation, where the focus is non-existent. Though it probably pisses you off, you shouldn’t be surprised. After all, it’s just simple economics, right?

—Mike Rothman

FireStarter: You Don’t Need Central Key Management

By Adrian Lane

If you are using encryption, somewhere you have encryption keys. Where you store them, and how they are managed and shared, are legitimate concerns. It is fashionable to store all keys in a single centralized key management server. Much as the name implies, this means storing all of your keys, of different types, for multiple use cases into a single key management server. Rich likes to call these ‘uber’ key manager, that handle any and all key functions; and are distinct from external key management servers that unify instances of single application, or provide key services across the disks in your storage array. Conceptually, a single product that handles all my key needs from a unified interface sounds great. But the real question is: why do you need it?

Central key management is not simplified key management. Central key management requires architectural and deployment changes. Consolidation of key storage and use policies does not ensure easier management, but does mean increased cost. Few people want or need centralized key management. Putting all your keys from every application or services into a single monolithic key management store offers few advantages, and creates a number of problems itself. The implication is that a central server will offer easier management, increased redundancy, and greater functionality; but these are often illusory benefits, based on solving a problem you did not have to begin with. In practice the internal and external key services that came with the products you already own are likely not only sufficient, but better. Here’s why:

  • No reason to share keys – Databases, disks, applications, file systems, and wherever else you encrypt seldom (if ever) need to share keys across these different services. Even if the encryption algorithm, key type, and key size are all consistent, there is no need to share keys between your tape drive and web application server. Using encryption to provide data integrity and privacy is a common goal, but the use cases and technical constraints are radically different.
  • Redundant – Why use it if it is already built into your application? Internal key management is built into most applications and cryptographic systems such as storage products and file-level encryption tools. External key management – for products that really need external support for good key security and proper separation of duties (SoD) – is provided by application libraries and database encryption products. Failover, backup, management interfaces, rotation, and cipher strength are all common features, so why centralize? Multiple services mean more interfaces to learn, but inherently provides SoD and focused policy management.
  • Cost – Central key management servers are standalone, dedicated products. They excel in areas such as key security, ease of use, key sharing, etc. But they are still an additional investment.
  • Policy Management – A single management console to manage the system sounds like a great convenience, but I don’t always want the same policies across dissimilar applications and use cases. In fact, I usually want different key lengths, different rotation schedules, and different ciphers, depending on the data I am protecting, and prefer the granularity to specify them at the level of an individual use case.
  • Single administration Console Having a single location to manage keys is conceptually useful. It may actually be useful if you have a very large number of users or must distribute keys and data across a large organization. Most of you reading this, however, work for small shops, and the one or two areas where you have deployed encryption do not require centralization. Few of you are at large enough organizations to worry about thousands of users each with hundreds of keys – and thus to need central key management to address data access issues across a dispersed environment.
  • Key rotation – Having a central key server to automate key management, especially the complexity of key rotation, is a common motivator for central services. Rotation, or key-cycling, is a common feature whereby key management products periodically issue new keys on the premise that with sufficient time and effort, someone will be able to discover the encryption keys in use. Theoretically you would issue a new key and then re-encrypt all data under the new key, but in practice it would take months of even years to re-encrypt everything, as the data sets are simply too large, and the media might even be off-line or off-site. In this scenario data is only re-encrypted under new keys opportunistically when it’s rewritten (perhaps also when it’s reread). But there is no guarantee that data will be re-encrypted. With every key rotation cycle, a new set of keys is generated, and the old ones must be retained to decrypt older data. Over time data will be encrypted under so many different keys that you must use a key manager just to keep track of what’s what. It’s a side effect of the encryption scheme, and for a modicum of extra security you get a bloated key server. Better to keep this compartmentalized than centralized.

Don’t go looking for central key management when external key management is all you need. Central key management is occasionally necessary – most often for existing systems with really bad built-in key management, geographically dispersed servers that require key sharing, or thousands of users each with multiple keys. A single point of management is veru much a secondary advantage, however, and should not drive your decisions. So why do you think you need it? What’s the advantage to you?

—Adrian Lane

Friday, April 16, 2010

ESF: Endpoint Incident Response

By Mike Rothman

Nowadays, the endpoint is the path of least resistance for the bad guys to get a foothold in your organization. Which means we have to have a structured plan and process for dealing with endpoint compromises. The high level process we’ll lay out here focuses on: confirming the attack, containing the damage, and then performing a post-mortem.

To be clear, incident response and forensics is a very specialized discipline, and hairy issues are best left to the experts. But that being said, there are things you as a security professional need to understand, to ensure the forensics guys can do their jobs.

Confirming the attack

There are lots of ways your spidey-sense should start tingling that something is amiss. Maybe it’s the user calling up and saying their machine is slow. Maybe it’s your SIEM detecting some weird log records. It could be your configuration management system reverting inexplicable changes or noting the presence of strange executables. Or perhaps your network flow analysis shows some reconnaissance activities from the device. A big part of the security management process is about being able to fire alerts when something suspicious is happening.

Then we make like bloodhounds and investigate the issue. We’ve got to find the machine and isolate it. Yes, that usually means interrupting the user and ‘inviting’ them to grab a cup of coffee, while you figure out what a mess they’ve made. The first step is likely to do a scan and compare with your standard builds (you remember the standard build, right?). Basically we look for obvious changes that cause issues.

If it’s not an obvious issue (think tons of pop-ups), then you’ve got to go deeper. This usually requires forensics tools, including stuff to analyze disks and memory to look for corruption or other compromise. There are lots of good tools – both open source and commercial – available for your forensics toolkit.

We do recommend you take a course in simple forensics as you get started, for a simple reason. You can really screw up an investigation by doing something wrong, in the wrong order, or using the wrong tools. If it’s truly an attack, your organization may want to prosecute at some point, and that means you have to maintain chain of custody on any evidence you gather. You should consult a forensics expert and probably your general counsel to identify the stuff you need to gather from a prosecution standpoint.

Containing the damage

“Houston, we have a problem…” Yup, your fears were justified and an endpoint or 200 have been compromised – so what to do? First off, you should inherently know what to do because you have a documented incident response plan, and you’ve practiced the process countless times, and your team springs into action without prompting, right? OK, this is the real world, so hopefully you have a plan and your team doesn’t look at you like an alien when you take it to DEFCON 4.

In all seriousness, you need to have an incident response plan. And you need to practice it. The time to figure out your plan stinks is not while a worm is proliferating through your innards at an alarming rate. We aren’t going to go into depth on that process (we’ll be doing a series later this year on incident response), but the general process is as follows:

  • Quarantine – Bad stuff doesn’t spread through osmosis – you need a network in place to allow malware to find new targets and spread like wildfire, so first isolate the compromised device. Yes, user grumpiness may follow, but whatever. They got pwned, so they can grab a coffee while you figure out how to contain the damage.
  • Assess – How bad is it? How far has it spread? What are your options to fix it? The next step in the process is to understand what you are dealing with. When you confirm the attack, you probably have a pretty good idea what’s going on. But now you have to figure out what the best option(s) is to fix it.
  • Workaround – Are there settings that can be deployed on the perimeter or at the network layer that can provide a short term fix? Maybe it’s blocking communication to the botnet’s command and control. Or possibly blocking inbound traffic on a certain port or some specific non-standard protocol that is the issue. Obviously be wary of the ripple effect of any workaround (what else does it break?), but allowing folks to get back to work quickly is paramount, so long as you can avoid the risk of further damage.
  • Remediate – Is it a matter of changing a setting or uninstalling some bad stuff? That would be optimistic, eh? Now is when you figure out how to fix the issue, and increasingly these days re-imaging is the best answer. Today’s malware hides so well it’s almost impossible to entirely inoculate a compromised device, and impossible to know you got it all. Which means part of your incident response plan should be a leveraged way to re-image machines.

At some point you have to figure out if this is an incident you can handle yourself, or if you need to bring in the artillery, in the form of forensics experts or law enforcement. Your IR plan needs to be identify scenarios which call for experts, and which call for the law. You don’t want that to be a judgement call in the heat of battle. So define the scenarios, establish the contacts (at both forensics firms and law enforcement), and be ready. That’s what IR is all about.

Post mortem

Once most folks get done cleaning up an incident, they think the job is done. Well, not so much. The reality is that the job has just begun, since you need to figure out what happened and make sure it doesn’t happen again. It’s OK to get nailed by something you haven’t seen before (fool me once, shame on you). It’s not OK to get nailed by the same thing again (fool me twice, shame on me). So you’ve got to schedule a post-mortem.

The post-mortem is not about laying blame – it’s about making sure it doesn’t happen again. So you need someone to candidly and in great detail understand what happened and where the existing defenses failed. Again, it is what it is and it’s critical that the organization can accept failures and move on. But not before you figure out whether process, controls, product, or people need to change.

By the way, it’s very hard to fight human nature and build a culture where failure is OK and post mortems are learning experiences, as opposed to a venue for everyone to cover their respective asses. But we don’t believe you can be successful at security without a strong incident response plan and that requires unemotional post-mortem analysis.

And with that, we come to the conclusion of the Endpoint Security Fundamentals series. We’ll be packaging it up in white paper form over the next week, and it will then be posted to the research library. As always, if there are things we missed or ideas you disagree with, please continue to contribute. Securosis research is an ongoing process, so things will change and we’ll update the documents as required.

Other posts in the Endpoint Security Fundamentals Series

  1. Introduction
  2. Prioritize: Finding the Leaky Buckets
  3. Triage: Fixing the Leaky Buckets
  4. Controls: Update and Patch
  5. Controls: Secure Configurations
  6. Controls: Anti-Malware
  7. Controls: Firewall, HIPS, and Device Control
  8. Controls: Full Disk Encryption
  9. Building the Endpoint Security Program
  10. Endpoint Compliance Reporting

—Mike Rothman

Public Goods

By Adrian Lane

Chris Pepper tweeted a very cool post on Why Content is a Public Good. The author, Milena Popova, provides an economist’s perspective on market forces and digital goods. Her premise is that in economic terms, many types of electronic content are “public goods” – that being a technical term for objects with infinite supply and no good way to control consumption. She makes the economic concepts of ‘rival’ and ‘excludable’ very easy to understand, and by breaking it down into rudimentary components, makes a compelling argument that content is a public good:

It means that old business models based on content being a club good simply don’t work. It means we have to rethink our relationship with content – as creators, as distributors and as consumers. It means that there are a lot of giants in the content distribution industry whose livelihoods (profit margins) are being pulled out from under them faster than they can say “illegal downloads”, and they are fighting it. Of course they’re fighting it. They’ve had an incredibly profitable business model for about a century and suddenly they don’t. Let’s face it, human beings don’t like change at the best of times, and we sure as hell don’t like it when it means less cash in our pockets.

I have written many posts on how economics affect DRM, RIAA, and ‘piracy’; and on the difference between actual security and security marketing, so I won’t rehash those subjects here; but note the common theme is that a busted business model is the root of the problem. Right now I want to stay away from some of the negativity of those posts, and instead focus on the economic drivers. Ms. Popova does a much better job than I of isolating the underlying forces, and discusses the factors in a way that helps us begin to visualize possible solutions.

A lot of people have a hard time with the concept of free and how you can actually make money in a world with so much free stuff. In a capitalist society we all have trouble with this. I talk to people in IT who still don’t think Linux and Java are viable technologies, and no one could make money with those products. But the availability of free stuff requires you to think a little differently about value – fewer people will pay money for the everyday and ordinary stuff because they don’t have to, but they will pay for things they perceive as special. In fact, I don’t think I fully grasped the concept and implications until I started working at Securosis. We are a research company that gives away most of our products for free, but charges for services and engagements.

One area I where was at odds with Popova was on the concept of “price discrimination”. From my perspective this looks more like the market being able to set the price, but do so far more efficiently: person to person, item by item, and adjusted over time. This is a very cool concept if you think about something like television: If you pay channel by channel, how many channels would you pay for? You have 400 or so, but I bet when it came to spending money, very few would get your hard-earned dollars. The NFL knows this, as football not only drives huge ad revenue, but single-handedly the bulk of hi-def television sales and additional add-on packages. If it was not for bundling into programming packages, many (most?) other channels would not be able to survive.

All in all, one of the better posts I have seen on the problems of dealing with consumer media.

—Adrian Lane

Friday Summary: April 16, 2010

By Adrian Lane

I am sitting here staring at power supplies and empty cases. Cleaning out the garage and closets, looking at the remnants from my PC building days. I used to love going out to select new motherboard and chipset combinations, hand-selecting each component to build just the right database server or video game machine. Over the years one sad acknowledgement needed to be made: after a year or so, the only pieces worth a nickel were the power supply and the case. Sad, but you spend $1,500.00 and after a few months the freaking box that housed the parts was the only remaining item of value.

I was thinking about this during some of our recent meetings with clients and would-be clients. Rich, Mike, and I are periodically approached by investors to review portfolio companies. We look both at their technology and market opportunities, and determine whether we feel the product is hitting the mark, and reaching buyers with the right product and message. We are engaged in response to either mis-aligned vision between investors and company operators (shocker, I know), or more commonly to give the investors some understanding of whether the company is worth salvaging through additional investment and a change of focus. Sometimes the company has followed market trends to preserve value, but quite a few turn out to be just a box of old parts. If this is not a direct consequence of Moore’s law, it should be.

We see cases where technologies were obsolete before the hit they market. In a handful of instances, we do find one or two worth salvaging, but not for the reasons they thought. In those cases the engineering staff was smart enough, or lucky enough, to build a deployment model or architecture that is currently relevant. The core technology? Forget it. Pitch it in the dust bin and take the write-off because that $20M investment is spent. But the ‘box’ was valuable enough to salvage, invest in, or sell. It’s ironic, and it goes to show how tough technology start-ups are to get right, and that luck is often better than planning.

On to the Summary:

Webcasts, Podcasts, Outside Writing, and Conferences

Favorite Securosis Posts

Other Securosis Posts

Favorite Outside Posts

Project Quant Posts

Top News and Posts

Blog Comment of the Week

Remember, for every comment selected, Securosis makes a $25 donation to Hackers for Charity. This week’s best comment goes to LonerVamp, in response to Security Incite: Just Think.

Nice opening post, especially with the kicker at the end that you’re actually writing it on a plane! :) I definitely find myself purposely unplugging at times (even if I’m still playing with something electronic) and protecting my private time when reasonable. I wonder if this ultimately has to do with the typically American concept of super-efficiency…milking every waking moment with something productive…at the expense of the great, relaxing, leisure things in life. I could listen to a podcast during this normally quiet hour in my day! And so on…
@Porky Risk…Pig: It doesn’t help that every security professional shotguns everyone else’s measurements…and with good reason! At some point, I think we’ll have to accept that every company CEO is different, and it takes a person to distill the necessary information down for her consumption. No tool will ever be enough, just like no tool ever determines if a product will be successful.
@My dad can beat up your dad: Ugh…I have some pretty strong opinions about cyberbullying and home internet monitoring…but I tend to shut up a bit because I don’t have kids and I’ll likely rub parents wrong (not that my humor isn’t dark enough to do that anyway!). While social networking didn’t exist when I was in school, I certainly have to give a lot of caution to taking away a child’s privacy. Protection is one thing, but being overly controlling and in their business is often a recipe for a lifetime of resentment…especially in their most carefree years of life where they have hours upon hours of free time without the burden of responsibilities or death anywhere on the horizon. I think it is simply most important to keep honest communication open (both ways) and make sure actions are taken when something awry happens. If someone isn’t your friend, why are you “friends” on Facebook with them? Masochistic? Sinister plans?
With rare, very saddening exceptions, a vast majority of kids will survive despite the rigors of childhood and school and self-discovery outside protective umbrellas. (I do make one exception. If there is monitoring and intrusion of privacy, they should never find out and you should never make them suspect it. It should be a silent angel watching over. But once you flaunt it or they find out, that is when the resentment and deeper hiding will happen…imo. Again, this is from a one-sided perspective and does not apply to all personalities. :) )

—Adrian Lane

Thursday, April 15, 2010

ESF: Endpoint Compliance Reporting

By Mike Rothman

You didn’t think we could get through an 11-part series about anything without discussing compliance, did you? No matter what we do from a security context – whatever the catalyst, budget center, or end goal – we need to substantiate implemented controls. We can grind out teeth and curse the gods all we want, but security investments are contingent on some kind of compliance driver.

So we need to focus on documentation and reporting for everything we do. Further, we discussed operational efficiencies in the security programs post, and the only way to get any kind of leverage from an endpoint security program is to automate the reporting.

Document what?

First we need to understand what needs to be documented from an endpoint perspective for the regulations/standards/guidance you deal with. You must be able to document the process/procedures of your endpoint program, as well as the data substantiating the controls. Process either exists or it doesn’t, so that documentation should be straightforward to produce.

On the other hand, figuring out which data types corroborate which controls and apply to which standards requires a big matrix to handle all the permutations and combinations. There are two ways to do this:

  1. Buy it – Many of the IT GRC tools out there (and don’t get us started on the value of IT GRC tools) help to manage the workflow of a compliance program. The key capability here is the built-in map, which connects technical controls to regulations, ostensibly so you don’t have to. But these tools cost money and provide limited value.
  2. Build it – The other option involves going through your regulations and figuring out relevant controls. This is about as fun as a root canal, but it has to be done. More likely, you can start with something your buddies, auditor, or vendors have. Vendors have excellent motivation to figure out how their products – representing a variety of security controls – map to the various regulations their customers need to address. The data is out there – you just have to assemble it.

Actually, there is a third option: to just license the content from an organization like the Unified Compliance Framework folks. They license a big-ass spreadsheet with the map, and their pricing is rather economical.


Now that you know what you need to report on, how do you do it? This question is bigger than your endpoint security program, and applies to every security program you run. We recommend you think architecturally. You’ve got certain domains of controls – think network, endpoint, data center, applications, etc. You want to put together a few things for each domain to make the auditor happy:

  • Control list – Go back to your control maps and make a big list of all the controls required for the auditor’s checklist (they all have checklists). Make sure the auditor buys into your list, and that you aren’t missing anything.
  • Logical architecture – Show graphically (a picture is worth a thousand words) how your controls are implemented. Right – every control on the list should appear on the logical architecture.
  • Data – You didn’t really think the auditor would just believe your architecture diagram, did you? Now you need the data from each of your systems (endpoint suite, configuration management, full disk encryption, etc.) to show that you’ve actually implemented the controls. Your vendor likely has a pre-built report for your regulation, so you shouldn’t have to do a lot of manual report generation.

To be clear, one of the value propositions of IT GRC and other compliance automation products like log management/SIEM is to aggregate all this information (not just from the endpoint program, but from all your programs) and spit out an integrated report. For the most part, with a bit of angst in deployment, these tools can help reduce the burden of preparing for frequent audits across multiple regulations for global enterprises. The question to answer is whether the tool can pay for itself in terms of saved time and effort – is the ROI sufficient?

Dealing with deficiencies

One other thing to consider is the reality of an audit pointing out a specific deficiency in your endpoint security program. The auditor/assessor is there to find problems, and likely they will. But that doesn’t mean the auditor is right.

Yes, we said it. Sometimes auditors take liberties and/or subjectively decide how to interpret a specific regulation. If there is a specific reason you decided to either bypass a control – or more likely, implement a compensating control – make your case.

In the event (however unlikely) there is a legitimate deficiency, you need to fix it. Welcome, Captain Obvious! During the next audit, first go through the list of previous deficiencies and how you’ve remediated them. Make a big deal of addressing the deficiencies, which will get the audit off on the right foot.

What’s next? We’ll cap off the Endpoint Security Fundamentals series by talking about incident response. Stay tuned for the exciting conclusion. ;-)

Other posts in the Endpoint Security Fundamentals Series

—Mike Rothman

Wednesday, April 14, 2010

ESF: Building the Endpoint Security Program

By Mike Rothman

Over the previous 8 posts in this Endpoint Security Fundamentals series, we’ve looked at the problem from the standpoint of what to do right awat (Prioritize and Triage) and the Controls (update software and patch, secure configuration, anti-malware, firewall, HIPS and device control, and full disk encryption). But every experienced security professional knows a set of widgets doesn’t make a repeatable process, and it’s really the process and the people that makes the endpoints secure.

So let’s examine how we take these disparate controls and make them into a program.

Managing Expectations

The central piece of any security program is making sure you understand what you are committing to and over-communicating your progress. This requires a ton of meetings before, during, and after the process to keep everyone on board. More importantly, the security team needs a standard process for communicating status, surfacing issues, and ensuring there are no surprises in task completion or results.

The old adage about telling them what you are going to do, doing it, and then telling them what you did, is absolutely the right way to handle communications. Forget the ending at your own peril.

Defining success

The next key aspect of the program is specifying a definition for success. Start with the key drivers for protecting the endpoints anyway. Is it to stop the proliferation of malware? To train users? To protect sensitive data on mobile devices? To improve operational efficiency? If you are going to spend time and money, or to allocate resources you need at least one clear driver / justification.

Then use those drivers to define metrics, and operationalize your process based on them. Yes, things like AV update efficiency and percentage of mobile devices encrypted are uninteresting, but you can trend off those metrics. You also can set expectations at the front end of the process about acceptable tolerances for each one.

Think about the relevant incident metrics. How many incidents resulted from malware? User error? Endpoint devices? These numbers have impact – whether good or bad. And ultimately it’s what the senior folks worry about. Operational efficiency is one thing – incidents are another.

These metrics become your dashboard when you are communicating to the muckety-mucks. And remember to use pie charts. We hear CFO-types like pie charts. Yes, I’m kidding.

User training

Training is the third rail of security, and needs discussion. We are fans of training. But not crappy, check-the-box-to-make-the-auditor-happy training. Think more like phishing internal users and, using other social engineering tactics to show employees how exposed they are. Good training is about user engagement. Unfortunately most security awareness training sucks.

Keep the goals in mind. The end user is the first line of defense (and for mobile professionals, unfortunately also the last) so we want to make sure they understand what an attack looks like and what to do if they think they might have a problem. They don’t have to develop kung fu, they just need to understand when they’ve gotten kicked in the head. For more information and ideas, check out Rich’s FireStarter from Monday.

Operational efficiencies

Certainly one of the key ways to justify the investment in any kind of program is via operational efficiencies, and in the security business that means automation whenever and wherever possible. So just think about the controls we discussed through this series, and how to automate them. Here’s a brief list:

  • Update and Patch, Secure Configurations – Tools to automate configuration management kill these two birds with one stone. You set a policy, and can thus both enforce standard configurations and keep key software updated.
  • Anti-malware, FW/HIPS – As part of the endpoint suites, enforcing policies on updates, software distribution and policy settings are fairly trivial. Here is the leverage (and the main justification) for consolidating vendors on the endpoint – just beware folks who promise integration, but fail to deliver useful synergy.
  • Device control, full disk encryption, application white listing – These technologies are not as integrated into the endpoint suites at this point, but as the technologies mature, markets consolidate, and vendors actually get out of their own way and integrate the stuff they buy, this will get better.

Ultimately, operational efficiencies are all about integrating management of the various technologies used to protect the endpoint.

Feedback loops

The other key aspect of the program is making sure it adapts to the dynamic nature of the attack space. Here are a few things you should be doing to keep the endpoint program current:

  • Test controls – We are big fans of hacking yourself, which means using hacking tools to test your own defenses. Check out tools like Metasploit and the commercial offerings, and send phishing emails to your employees trying to get them to visit fake sites – which presumably would pwn them. This is a critical aspect of the security program.
  • Endpoint assessment – Figure out to what degree your endpoints are vulnerable, usually by scanning devices on connect with a NAC-type product, or with a scanner. Look for patterns to identify faulty configuration, ineffective patching, or other gaps in the endpoint defenses.
  • Configuration updates – A couple times a year new guidance emerges (from CIS and NIST, etc.) recommending changes to standard builds. Evaluating those changes, and figuring out whether and how the builds should change, helps to ensure the endpoint protection is always adapted to current attacks.
  • User feedback – Finally, you need to pay attention to the squeaky wheels in your organization. Well, not entirely, but you do have to factor in whether they are complaining about draconian usage policies – and more importantly whether some the controls are impeding their ability to do their jobs. That’s what we are trying to avoid.

The feedback loops will indicate when it’s time to revisit the controls, perhaps changing standard builds or considering new controls or tools. Keep in mind that without process and failsafes to keep the process relevant, all the technology in the world can’t help.

We’ll wrap up the series in the next two posts by discussing the compliance reporting requirements of the endpoint security program, and then endpoint-centric incident response.

Other posts in the Endpoint Security Fundamentals Series

—Mike Rothman

Incite 4/14/2010: Just Think

By Mike Rothman

As numb as we are to most advertising (since we are hit with thousands of advertising exposures every day), sometimes an ad campaign is memorable and really resonates. No, seeing Danica Patrick on a massage table doesn’t qualify. But Apple’s Think Different campaign really did. At that point, Apple was positioning to the counter-culture, looking for folks who didn’t want to conform. Those who had their own opinions, but needed a way to set them loose on the world.

Now that is definitely thinking different... Of course, we all want to think we are more than just cogs in the big machine and that we do matter. So the campaign resonated.

But nowadays I’m not so worried about thinking differently, but just thinking at all. You see, we live in a world of interruption and multitasking. There is nowhere to hide any more, not even at 35,000 feet.

Flying used to be my respite. 2-5 hours of solitude. You know, put in the ear buds, crank up the iPod, and tune out. Maybe I’d catch up on some writing or reading. Or even at the risk of major guilt, I’d get some mental floss (I’m plowing through the Daniel Silva series now) and crank through some fiction on the flight. Or Lord help me, sometimes I’d just sit and think. An indulgence I don’t partake in nearly enough during my standard routine.

Yet through the wonders of onboard WiFi, you can check email, surf the Web, tweet to your friends (yo, dog, I’m at 30K feet - and you are not!) or just waste time. All for $9.95 per flight. What a bargain.

And if you absolutely, positively need to send that email somewhere over Topeka, then the $9.95 is money well spent. Yet in reality, I suspect absolutely, positively means when you get to your destination.

What you don’t see is the opportunity cost of that $9.95. Not sure you can put a value on spending 3 hours battling spies or catching up on some journaling or even revisiting your plans for world domination. On my way back from RSA, I did use the on-board WiFi and to be honest it felt like a piece of me died. I was pretty productive, but I didn’t think, and that upset me. The last bastion of solitude was gone, but certainly not forgotten.

So yes, I’m writing this from 34,701 feet somewhere above New Mexico. But my battery is about dead, and that means it’s time to indulge. There are worlds to dominate, windmills to chase, strategies to develop, and I can’t do that online. Now quiet down, I need to think…

– Mike.

PS: Good luck to AndyITGuy, who is leaving his ATL digs to head to Cincy. Hopefully he’ll keep writing and plug into the security community in Southern Ohio.

Photo credits: “think___different” originally uploaded by nilson

Incite 4 U

  1. Porky, Risk Management, Pig – Let’s all welcome Jack Jones back to bloggy land, and he restarts with a doozy – basically saying risk management tools are like putting lipstick on a pig. Being a vegetarian and thinking about actually putting lipstick on a pig, I can only think of the truism from Jules in Pulp Fiction, “we’d have to be talkin’ about one charming motherf***ing pig.” But I’ll summarize Jack’s point more succinctly. Garbage In = Garbage Out. And even worse, if the analysis and the outcomes and the quantification are lacking, then it’s worse than garbage out: it’s sewage out. But senior management wants a number when they ask about risk, and the weak security folks insist on giving it to them, even though it’s pretty much arbitrary. Off soapbox now. – MR

  2. Craponomics – Repeat after me – it’s all about the economics. (I’m starting to wish I took one of those econ classes in school). According to the New York Times, lenders sort of ignore many of the signs of ID theft because they’d rather have the business. The tighter the fraud controls, the fewer people (legitimate and criminal) they can lend to, and the lower the potential profits. It’s in their interest to tolerate a certain level of fraud, even if that hurts ID theft victims. Remember, the lenders are out to protect themselves, not you. Can you say moral hazard? – RM

  3. Partly Paranoid, with a chance of PR – From what I hear, Google is now paranoid about security. Call me a cynic, but when someone trashes your defenses, is your response to use more web-based computing products and services, like Google Chrome? I am sure they are thinking they’ll modernize their defenses with Chrome, and all those old threat vectors will be magically corrected. You know, like XSS and injection attacks. I am thinking, “Someone broke into my company, now Chrome’s source code is suspect until I can prove attackers did not gain access to the source code control system.” Give Google credit for disclosure, and odds are they will be more secure with Chrome, but that was just a stepping stone in the process. I am far more interested in the steps taken to provide redundant security measures and perhaps some employee education on anti-phishing and security. Something that helps after an employee’s browser is compromised. There will always be another browser hack, so don’t tell me the answer is Chrome. Blah. – AL

  4. Yes, you are an addict… – It was funny to read Chris Nickerson’s post on FUDSEC about being a security addict, especially since that’s the entire premise of the Pragmatic CSO. But we look at the problem from different perspectives. Chris is right in pointing out that although we security folks tend to be powerless, that doesn’t mean we are helpless. It’s an important nuance. Personally I found his 12 steps lacking, especially compared to mine. But he also was working within the context of one blog post, and I wrote a book. All kidding aside, there are things we can control and things we can’t. Security is a hard job and you have to have the right mindset to survive. Whether you subscribe to Chris’ 12-step philosophy or mine or none at all, realizing that you do what you can will make all the difference to your happiness in security. – MR

  5. Accidental Developer – Marisa Fagan of Errata Security conducted a useful survey a few months back on security and software development. When I was taking the survey, the question “What Software Development Methodology do you follow” had me thinking “I don’t”. Sure enough, “Ad-hoc” was the most popular answer. Does this surprise you? I think the popularity of that choice is a good illustration of how early we are in the evolution of software security. I have never followed anyone’s secure development cycle, as I haven’t yet seen a framework I was interested in until recently. This will change with the work coming out of Microsoft and other firms as templates for good development practices. I think that with the industry settling into the – dare I say it – “trough of disillusionment” with Agile, it’s going to take a while before Agile is sufficiently developed to embrace widely accepted principles for secure code development. What cracked me up when taking the survey was, looking for the “Ad-Hoc” check box, I saw “Securosis Secure SDL” as an option. I asked Rich if we had ever written a secure software development lifecycle, to which he responded something like “Yeah, stupid, you did.” I went back and searched the blog and, sure enough, I am stupid. Out of frustration at the lack of practical guidance I had cobbled together a rudimentary process, but not fully fleshed it out. I think it’s time to take a fresh look at the process, and come up with some applied practices for resilient code within an Agile methodology. – AL

  6. Why sell it when you can give it away? – In the endpoint fundamentals series, I talked about anti-malware and touched on free AV. In the corporate context, you get what you pay for given the non-existent advanced detection techniques and management. But from a consumer perspective, is free AV such a bad thing? Not if the advanced stuff you want is there, which makes Check Point’s one-day give-away of ZoneAlarm and an identity service from Intersections, timed to coincide with Patch Tuesday, interesting. On the other hand, this reeks of desperation from CHKP, which can’t seem to make inroads into the endpoint space. Anyhow, if you’re looking for a full endpoint suite for your mother-in-law’s computer, have at it – even if you like your mother-in-law. – MR

  7. Hardening the Cloud – Yesterday I was talking with an investment friend about how I expect to see big hardware refreshes marching hand in hand with cloud computing adoption. It turns out there are some problems related to cloud computing security that are difficult or impossible to solve in software alone. Chris Hoff discusses Trusted Execution Technology, which basically requires new hardware and software, some of which isn’t available yet. (TXT ideally allows you to only execute a VM on hardware that passes assurance checks). There are also memory protection enhancements important to cloud computing that will require hardware upgrades. As Chris says, it doesn’t look like any public cloud providers are using these boxes yet, but make sure you take them into account when planning internal deployments. – RM

  8. My Dad can beat up your Dad – I remember those childish elementary school taunts like it was yesterday. I always thought my Dad was pretty tough, but I also knew he carried heat when he was working in pharmacies in “undesirable areas”. Hard to outrun a bullet. But the rules have changed now, and the bullies (and their parents) who my kids will have to deal with are different in nature. Physical bullies I can deal with, but the email (and Facebook) gladiators are much tougher. Some tactics are discussed on the McAfee blog, as well as in NetworkWorld (featuring our buddy Martin McKeay). They have good ideas there, but I’m still a fan of monitoring – at least to provide an audit trail in case things do go south. And also education – I spend a lot of time teaching my kids wrong from right, so why wouldn’t I do the same for their Internet use? – MR

—Mike Rothman

Tuesday, April 13, 2010

ESF: Controls: Full Disk Encryption

By Mike Rothman

It happens quickly. An end user just needed to pick up something at the corner store or a big box retailer. He was in the store for perhaps 15 minutes, but that was plenty of time for a smash and grab. And then your phone rings, a laptop is gone, and it had information on about 15,000 customers. You sigh, hang up the phone and call the general counsel – it’s disclosure time.

Sound familiar? Maybe this has been you. It likely will be, unless you proactively take action to make sure that the customer data on those mobile devices cannot be accessed by whoever buys the laptop on the gray market. That’s right, you need to deploy full disk encryption (FDE) on the devices. Unless you enjoy disclosure and meeting with lawyers, that is.


Ultimately, encryption isn’t very novel. But managing encryption across an enterprise is, so key management and ease of use end up being the key features that generally drive FDE. As we’ve harped throughout this series, integration of that management with the rest of the endpoint functions is critical to gaining leverage and managing all the controls implemented on the endpoints.

Of course, that’s looking at the issue selfishly from the security professional’s perspective. Ultimately the success of the deployment depends on how transparent it is to users. That means it needs to fit in with the authentication techniques they already use to access their laptops. And it needs to just work. Locking a user out of their data, especially an important user at an inopportune, time will make you a pretty unpopular person.

Finally, don’t forget about those backups or software updates. If your encryption breaks your backups (and you are backing up all those laptops, right?) it’s a quick way to find yourself in the unemployment line. Same goes for having to tell the CIO everyone needs to bring their laptops back to the office every Patch Tuesday to get those updates installed.

Integration with Endpoint Suites

Given the natural order of innovation and consolidation, the industry has seen much consolidation of FDE solutions by endpoint vendors. Check Point started the ball rolling by acquiring Pointsec; shortly afterwards Sophos acquired Utimaco and McAfee acquired SafeBoot, which of course gives these vendors the ability to bundle FDE with their endpoint suites.

Now bundling on the purchase order is one thing, but what we are really looking for is bundling from a management standpoint. Can the encryption keys be managed by the endpoint security management console? Is your directory supported natively? Can the FDE policies be set up from the same interface you use for host firewalls and HIPS policies? Unless this level of integration is available, there is little leverage in using FDE from your endpoint vendor.

Free (as in beer?)

Like all good innovations, the stand-alone companies get acquired and then the capability tends to get integrated into the operating system – which is clearly the case with FDE. Both Microsoft BitLocker and Apple FileVault provide the capability to encrypt at the operating system level (Bitlocker is full drive, FileVault is OS). Yes, it’s free, but not really. As mentioned above, encryption isn’t really novel anymore, it’s the management of encryption that makes the difference. Neither Microsoft nor Apple currently provides adequate tools to really manage FDE across an enterprise.

Which means there will remain a need for third party managed FDE for the foreseeable future, and that also means the endpoint security suite is the best place to manage it. We expect further integration of FDE into endpoint security suites, further consolidation of the independent vendors, and ultimately commoditization of the capability. So we’ll joke over beers in a few years about how you use to pay separately for full disk encryption.

Now that we’ve examined the controls we use to protect the endpoints, we need to build a systematic program to ensure these controls are deployed, enforced, and reported on. That’s our topic for the next two posts, as we build the endpoint security program also consider what kind of reporting we need to keep the auditors happy.

Other posts in the Endpoint Security Fundamentals Series

—Mike Rothman

Monday, April 12, 2010

ESF: Controls: Firewalls, HIPS, and Device Control

By Mike Rothman

Popular perception of endpoint security revolves around anti-malware. But they are called suites for a reason – other security components ship in these packages, which provide additional layers of protection for the endpoint. Here we’ll talk about firewalls, host intrusion prevention, and USB device control.


We know what firewalls do on the perimeter of the network: selectively block traffic that goes through gateways by port and protocol. The functionality of a host firewall on an endpoint is similar. They allow an organization to enforce a policy governing what traffic the device can accept (ingress filtering) and transmit (egress filtering).

Managing the traffic to and from each endpoint serves a number of purposes, including hiding the device from reconnaissance efforts, notifying the user or administrators when applications attempt to access the Internet, and monitoring exactly what the endpoints are doing. Many of these capabilities are available separately on the corporate network, but when traveling or at home and not behind the corporate perimeter, the host firewall is the first defense against attacks.

Of course, a host firewall (like everything else that runs on an endpoint) takes up resources, which can be a problem on older or undersized machines. It also bears consideration that alerts multiply, especially when you have a couple thousand endpoints forwarding them to a central console, so some kind of automated alert monitoring becomes critical.

Although pretty much every vendor bundles a host firewall with their endpoint suite nowadays, the major operating systems also provide firewall options. Windows has included a firewall since XP, but keep in mind that the XP firewall does not provide egress (outbound) filtering – remedied in Windows Vista. Mac OS X 10.5 Leopard added a ‘socket’ firewall to manage application listeners (ingress), and deprecated the classic ipfw network firewall, which is still available.

As with all endpoint capabilities, just having the feature isn’t enough, since the number of endpoints to be managed puts a real focus on managing the policies. This makes policy management more important than firewall engine details.

Host Intrusion Prevention Systems (HIPS)

We know what network intrusion detection/prevention products do, in terms of inspecting network traffic and looking for attacks. Similarly, host intrusion detection/prevention capabilities look for attacks by monitoring what’s happening on the endpoint. This can include application behavior, activity logs, endpoint network traffic, system file changes, Windows registry changes, processes and/or threads, memory allocation, and pretty much anything else.

The art of making host intrusion prevention work is to set up the policies to prevent malware infection, without badly impacting the user experience or destroying the signal-to-noise ratio of alerts coming into the management console. Yes, this involves tuning, so you start with the product’s default settings (hopefully on a test group) and see what works and what doesn’t. You should be able to quickly optimize the policy.

Given the number of applications and activities at each endpoint, you can go nuts trying to manage these policies, which highlights the importance of the standard builds (as described in Controls: Secure Configurations). Starting with 3-4 different policies, and then you can manage others by exception. Keep in mind that tuning the product for servers is totally different, as the policies will need to be tailored for very different applications running on servers.

Currently, all the major endpoint suites include simple HIPS capabilities. Some vendors also offer a more capable HIPS product – typically targeting server devices, which are higher profile targets and subject to different attacks.

USB Device Control

Another key attack vector for both data compromise and malware proliferation is the USB ports on endpoint devices. In the old days, you’d typically know when someone brought in a huge external drive to pilfer data. Nowadays many of us carry a 16GB+ drive at all times (yes, your smartphone is a big drive), so we’ve got to control USB ports to address this exposure.

Moreover, we’ve all heard stories of social engineers dropping USB sticks in the parking lot and waiting for unsuspecting employees to pick them up and plug them in. Instant pwnage 4U! So another important aspect of protecting endpoints includes defining which devices can connect to a USB port and what those devices can do.

This has been a niche space, but as more disclosure is required for data loss, organizations are getting more serious about managing their USB ports. As with all other endpoint technologies, device control adds significant management overhead for keeping track of all the mobile devices and USB sticks, etc. The products in the space include management consoles to ease the burden, but managing thousands of anything is non-trivial.

Right now device control is a discrete function, but we believe these niche products will also be subsumed into the endpoint suites over the next two years. In the meantime, you may be able to gain some leverage by picking a device control vendor partnered with your endpoint suite provider. Then you should at least be able centralize the alerts, even if you don’t get deeper management integration.

Management Leverage

Though we probably sound like a broken record at this point, keep in mind that each additional security application/capability (control) implemented on the endpoint devices increases the management burden. So when evaluating technology for implementation, be sure to assess the additional management required and the level of integration with your existing endpoint management workflow.

We’ll wrap up our discussion of Endpoint Controls in the next post, as we discuss full disk encryption, which disclosure laws have shifted from nice-to-have to something you need deployed – immediately.

Other posts in the Endpoint Security Fundamentals Series

—Mike Rothman