Research

A recent post tying segmented web browsing to DMZs by Daniel Miessler got me thinking more about the network segmentation that is lacking in most organizations. The concept behind that article is to establish a browser network in a DMZ, wherein nothing is trusted. When a user wants to browse the web, the article implies that the user fires up a connection into the browser network for some kind of proxy out onto the big, bad Internet. The transport for this connection is left to the user’s imagination, but it’s easy to envision something along the lines of Citrix Xenapp filling this gap. Fundamentally this may offset some risk initially, but don’t get too excited just yet. First let’s clear up what a DMZ should look like conceptually. From the perspective of most every organization, you don’t want end users going directly into a DMZ. This is because, by definition, a DMZ should be as independent and self-contained as possible. It is a segment of your network that is Internet facing and allows specific traffic to (presumably) external, untrusted parties. If something were to go wrong in this environment, such as a server being compromised, the breach shouldn’t expose the internal organization’s networks, servers, and data; and doesn’t provide the gold mine of free reign most end users have on the inside. The major risk of the DMZ network architecture is an attacker poking (or finding) holes and building paths back into your enterprise environment. Access from the inside into the primary DMZ should be restricted solely to some level of bastion hosts, user access control, and/or screened transport regimens. While Daniel’s conceptual diagram might be straightforward, it leaves out a considerable amount of magic that’s required to scale in the enterprise, given that the browser network needs to be segregated from the production environments. This entails pre-authenticating a user before he/she reaches the browser network, requiring a repository of internal user credentials outside the protected network. There must also be some level of implied trust between the browser network and that pre-authentication point, because passing production credentials from the trusted to untrusted network would be self-defeating. Conversely, maintaining a completely different user database (necessarily equivalent to the main non-DMZ system, and possibly including additional accounts if the main system is not complete) is out of the question in terms of scalability and cost (why build it once when you can build it twice, at twice the price?), so at this point we’re stuck in an odd place: either assuming more risk of untrusted users or creating more complexity – and neither is a good answer. Assuming we can get past the architecture, let’s poke at the browser network itself. Organizations like technologies they can support, which costs money. Since we assume the end user already has a supported computer and operating system, the organization now has to take on another virtual system to allow them to browse the Internet. Assuming a similar endpoint configuration, this roughly doubles the number of supported systems and required software licenses. It’s possible we could deploy this for free (as in beer) using an array of open source software, but that brings us back to square one for supportability. Knock knock. Who’s there? Mr. Economic Reality here – he says this dog won’t hunt. What about the idea of a service provider basically building this kind of browser network and offering access to it as part of a managed security service, or perhaps as part of an Internet connectivity service? Does that make this any more feasible? If an email or web security service is already in place, the user management trust issue is eliminated since the service provider already has a list of authorized users. This also could address the licensing/supportability issue from the enterprise’s perspective, since they wouldn’t be licensing the extra machines in the browser network. But what’s in it for the service provider? Is this something an enterprise would pay for? I think not. It’s hard to make the economics work, given the proliferation of browsers already on each desktop and the clear lack of understanding in the broad market of how a proxy infrastructure can increase security. Supportability and licensing aside, what about the environment itself? Going back to the original post, we find the following: Browsers are banks of virtual machines Sandboxed Constantly patched Constantly rebooted No access to the Internet except through the browser network Untrusted, just like any other DMZ Here’s where things start to fall apart even further. The first two don’t really mean much at this point. Sandboxing doesn’t buy us anything because we’re only using the system for web browsing anyway, so protecting the browser from the rest of the system (which does nothing but support the browser) is a moot point. To maintain a patch set makes sense, and one could argue that since the only thing on these systems is web browsing, patching cycles could be considerably more flexible. But since the patching is completely different than inside our normal environment, we need a new process and a new means of patch deployment. Since we all know that patching is a trivial slam-dunk which everyone easily gets right these days, we don’t have to worry, right? Yes, I’m kidding. After all, it’s one more environment to deal with. Check out Project Quant for Patch Management to understand the issues inherent to patching anything. But we’re not done yet. You could mandate access to the Internet only through the browser network for the general population of end users, but what about admins and mobile users? How do we enforce any kind of browser network use when they are at Starbucks? You cannot escape exceptions, and exceptions are the weakest link in the chain. My personal favorite aspect of the architecture is that the browser network should be considered untrusted. Basically this means anything that’s inside the browser network can’t be secured. We therefore assume something in the browser network is always compromised. That reminds me of those containment units the Ghostbusters used to put Slimer

In the last Network Security Fundamentals post, we talked about monitoring (almost) everything and how that drives a data/log aggregation and collection strategy. It’s great to have all that cool data, but now what? That brings up the ‘C word’ of security: correlation. Most security professionals have tried and failed to get sufficient value from correlation relative to the cost, complexity, and effort involved in deploying the technology. Understandably, trepidation and skepticism surface any time you bring up the idea of real-time analysis of security data. As usual, it comes back to a problem with management of expectations. First we need to define correlation – which is basically using more than one data source to identify patterns because the information contained in a single data source is not enough to understand what is happening or not enough to make a decision on policy enforcement. In a security context, that means using log records (or other types of data) from more than one device to figure out whether you are under attack, what that attack means, and the severity of attack. The value of correlation is obvious. Unfortunately networks typically generate ten of thousands data records an hour or more, which cannot be analyzed manually. So sifting through potentially millions of records and finding the 25 you have to worry about represents tremendous time savings. It also provides significant efficiencies when you understand threats in advance, since different decisions require different information sources. The technology category for such correlation is known as SIEM: Security Information and Event Management. Of course, vendors had to come along and screw everything up by positioning correlation as the answer to every problem in security-land. Probably the cure for cancer too, but that’s beside the point. In fairness, end users enabled this behavior by hearing what they wanted. A vendor said (and still says, by the way) they could set alerts which would tell the user when they were under attack, and we believed. Shame on us. 10 years later, correlation is achievable. But it’s not cheap, or easy, or comprehensive. But if you implement correlation with awareness and realistic expectations, you can achieve real value. Making Correlation Work 4 U I liken correlation to how an IPS can and should be used. You have thousands of attack signatures available to your IPS. That doesn’t mean you should use all of them, or block traffic based on thousands of alerts firing. Once again, Pareto is your friend. Maybe 20% of your signatures should be implemented, focusing on the most important and common use cases that concern you and are unlikely to trigger many false positives. The same goes for correlation. Focus on the use cases and attack scenarios most likely to occur, and build the rules to detect those attacks. For the stuff you can’t anticipate, you’ve got the ability to do forensic analysis, after you’ve been pwned (of course). There is another more practical reason for being careful with the rules. Multi-factor correlation on a large dataset is compute intensive. Let’s just say a bunch of big iron was sold to drive correlation in the early days. And when you are racing the clock, performance is everything. If your correlation runs a couple days behind reality, or if it takes a week to do a forensic search, it’s interesting but not so useful. So streamlining your rule base is critical to making correlation work for you. Defining Use Cases Every SIEM/correlation platform comes with a bunch of out-of-the-box rules. But before you ever start fiddling with a SIEM console, you need to sit down in front of a whiteboard and map out the attack vectors you need to watch. Go back through your last 4-5 incidents and lay those out. How did the attack start? How did it spread? What data sources would have detected the attack? What kinds of thresholds need to be set to give you time to address the issue? If you don’t have this kind of data for your incidents, then you aren’t doing a proper post-mortem, but that’s another story. Suffice it to say 90% of the configuration work of your correlation rules should be done before you ever touch the keyboard. If you haven’t had any incidents, go and buy a lottery ticket – maybe you’ll hit it big before your number comes up at work and you are compromised. A danger of not properly defining use cases is the inability to quantify the value of the product once implemented. Given the amount of resources required to get a correlation initiative up and running, you need all the justification you can get. The use cases strictly define what problem you are trying to solve, establish success criteria (in finding that type of attack) and provide the mechanism to document the attack once detected. Then your CFO will pipe down when he/she wants to know what you did with all that money. Also be wary of vendor ‘consultants’ hawking lots of professional service hours to implement your SIEM. As part of the pre-sales proof of concept process, you should set up a bunch of these rules. And to be clear, until you have a decent dataset and can do some mining using your own traffic, paying someone $3,000 per day to set up rules isn’t the best use of their time or your money. Gauging Effectiveness Once you have an initial rule set, you need to start analyzing the data. Regardless of the tool, there will be tuning required, and that tuning takes time and effort. When the vendor says their tool doesn’t need tuning or can be fully operational in a day or week, don’t believe them. First you need to establish your baselines. You’ll see patterns in the logs coming from your security devices and this will allow you to tighten the thresholds in your rules to only fire alerts when needed. A few SIEM products analyze network flow traffic and vulnerability data as well, allowing you to use that data to make your rules smarter

So it’s probably apparent that Mike and I have slightly different opinions on some security topics, such as Monitoring Everything (or not). But sometimes we have exactly the same viewpoint, for slightly different reasons. Correlation is one of these later examples. I don’t like correlation. Actually, I am bitter that I have to do correlation at all. But we have to do it because log files suck. Please, I don’t want log management and SIEM vendors to get all huffy with that statement: it’s not your fault. All data sources for forensic information pretty much lack sufficient information for forensics, and deficient in options for tuning and filtering capabilities to make them better. Application developers did not have security in mind when they created the log data, and I have no idea what the inventors of Event Log had in mind when they spawned that useless stream of information, but it’s just not enough. I know that this series is on network fundamentals, but I want to raise an example outside of the network space to clearly illustrate the issue. With database auditing, the database audit trail is the most accurate reflection of the database transactional history and database state. It records exactly what operations are performed. It is a useful centerpiece of auditing, but it is missing critical system events not recorded in the audit trail, and it does not have the raw SQL queries sent to the database. The audit trail is useful to an extent, but to enforce most policies for compliance or to perform a meaningful forensic investigation you must have additional sources of information (There are a couple vendors out there who, at this very moment, are tempted to comment on how their platform solves this issue with their novel approach. Please, resist the temptation). Relational database platforms do a better job of creating logs than most networks, platforms, or devices. Log file forensics are a little like playing a giant game of 20 questions, and each record is the answer to a single question. You find something interesting in the firewall log, but you have to look elsewhere to get a better idea of what is going on. You look at an access control log file, and now it really looks like something funny is going on, but now you need to check the network activity files to try to estimate intent. But wait, the events don’t match up one for one, and activity against the applications does not map one-to-one with the log file, and the time stamps are skewed. Now what? Content matching, attribute matching, signatures, or heuristics? Which data sources you select depends on the threats you are trying to detect and, possibly, react to. The success of correlation is largely dependent on how well you size up threats and figure out which combination of log events are needed. And which data sources you choose. Oh, and then how well develop the appropriate detection signatures. And then how well you maintain those policies as threats morph over time. All these steps take serious time and consideration. So do you need correlation? Probably. Until you get something better. Regardless of the security tool or platform you use for threat detection, the threat assessment is critical to making it useful. Otherwise you are building a giant Cartesian monument to the gods of useless data. Share:

You may not know it, but lots of folks you know are zombies. It seems that life has beaten them down, and miraculously two weeks later they don’t say ‘hi’ – they just give you a blank stare and grin as the spittle drips out of the corners of their mouths. Yup, a sure sign they’ve been to see Dr. Feelgood, who heard for an hour how hard their lives are, and as opposed to helping to deal with the pain, they got their friends Prozac, Lexapro, and Zoloft numb it. These billion dollar drugs build on the premise that life is hard, so it’s a good idea to take away the ability to feel because it hurts too much. Basically we, as a society, are increasingly becoming comfortably numb. I’m not one to be (too) judgmental about the personal decisions that some folks make, but this one gets in my craw. My brother once said to me “Life is Pain,” and there is some truth to that statement. Clearly life is hard right now for lots of folks and I feel for them. But our society values quick fixes over addressing the fundamental causes of issues. Just look at your job. If someone came forward with a widget that would get you compliant, you’d buy it. Maybe you already have. And then you realize: there are no short cuts. You’ve got to do the work. Seems to me we don’t do the work anymore. Now, to be clear, some folks are ill and they need anti-depressants. I’ve got no issue with that – in fact I’m thankful that these folks have some options to lead normal lives and not hurt themselves and/or others. It’s the soccer mom (or dad) who is overwhelmed with having to get the kid’s homework done and getting them to baseball practice. That doesn’t make sense to me. I know it’s easier to take a pill than to deal with the problem, but that doesn’t make the problem go away. I guess that’s easy for me to say because thankfully I don’t suffer from depression. Yet, to come clean I spent most of my 20’s medicating in my own way. I got hammered every weekend and sometimes during the week. If I had invested in the market half of what I spent on booze, I wouldn’t be worrying about the mortgage. But I guess that I worry at all about anything is a good sign. Looking back, I was trying to be someone different – the “party guy,” who can drink beer funnels until he pukes and then drink some more. I was good at that. Then I realized how unfulfilling that lifestyle was for me, especially when the doctor informed me I had the liver of a 50 year old. Which is not real good when you are 30. Ten years later, I actually enjoy the ups and downs. OK, I like the ups more than the downs, but I understand that without feeling bad, I can’t appreciate when things are good. I’m getting to the point where I’m choosing what to get pissed off about. And I do still get pissed. But it’s not about everything and I get past my anger a lot faster. Basically, I’m learning how to let it go. If I can’t control it and I didn’t screw it up, there isn’t much I can do – so being pissed off about it isn’t helping anyone. By the way, that doesn’t mean I’m a puritan. I still tip back a few per week and kick out the jams a few times a year. The funnel is still my friend. The difference is I’m not running away from anything. I’m not trying to be someone else. I’m getting into the moment and having fun. There is a big difference. – Mike Photo credit: “Comfortably Numb” originally uploaded by Olivander Incite 4 U One of the advantages of working on a team is that we cover for each other and we are building a strong bench. This week contributor David Mortman put together a couple of pieces. Mort went and got a day job, so he’s been less visible on Securosis, but given his in-depth knowledge of all things (including bread making), we’ll take what we can get. I also want to highlight a post by our “intern” Dave Meier on Misconceptions of a DMZ, in which he dismantles a thought balloon put out there regarding virtualized web browser farms. Meier lives in the trenches every day, so he brings a real practitioner’s perspective to his work for Securosis. It’s About the Boss, Not the Org Chart – My buddy Shack goes on a little rampage here listing the reasons why security shouldn’t report to IT. I’m the first to think in terms of absolutes (the only gray in my life is my hair), but Dave is wrong here. I’m not willing to make a blanket statement about where security should report because it’s more about being empowered than it is about the org chart. If the CIO gets it and can persuade the right folks to do the right thing and support the mission, then it’s all good. If that can happen in the office of the CFO or even CEO, that’s fine too. Dave brings up some interesting points, but unless you have support from the boss, none of it means a damn thing. – MR Rock Stars Are a Liability – It looks like Forrester Research now requires all analysts to shut down their personal blogs, and only blog on the Forrester platform. I started Securosis (the blog) back when I was still working at Gartner, and took advantage of the grey area until they adopted an official policy banning any coverage of IT in personal blogs. That wasn’t why I left the company, but I fully admit that the reception I received while blogging gave me the confidence to jump out there on my own. In a big analyst firm the corporate brand is more