Research

We have spent much of this series on why benchmarking is important. But we also need to point out some situations where benchmarking may not be appropriate. There are clearly situations where you can’t benchmark, particularly is on granular operational data, which I call Ninja Metrics. Dependency: Peer Group Data Most organizations have ‘nascent’ metrics programs, which may actually be too kind. But not all. Some have embraced detailed programs that gathers all sorts of data, mostly focused on operations. This represents the next step of a metrics program, and can be represented by some of the ideas put forth through our Quant research projects. We have created highly granular process maps (with associated metrics) for Patch Management, Network Security Operations, and Database Security. Each report specifies 50+ distinct metrics you can measure for that discipline. Yes, they are comprehensive. But there is a clear issue regarding benchmarking at this level. You will have a hard time finding similarly granular data from other companies for comparison. So the key dependency in implementing a benchmarking effort is the availability of peer group data for comparison. Compare to Yourself What do world class athletes do when they reach the top of the heap? You know, folks like Michael Phelps, who has basically shattered every record there is to shatter. They start comparing themselves to their past performance. Improvement is measured internally rather than externally. Even if no one else has ever done better, you know you can. And this is what you will likely need to do the most granular operational functions. When you take a step back this makes a lot of sense. The reality is that you aren’t necessarily trying to ‘win’ relative to operational excellence. You want to improve. That said, it is important to have an idea of where you stand in comparison to everybody else, at least on the high-level operational metrics. But for the most granular metrics, not so much. We hope that over time enough companies will start tracking granular operational metrics, and become comfortable enough with benchmarking, to share their data. But that’s not going to happen tomorrow or even the day after. In the meantime you can (and should) continue to push your metrics program forward – just understand your comparisons may need to be internal. As we wrap up the Benchmarking series, we’ll look at how to get some Quick Wins and see the process in action. Share:

I love it when people froth at the mouth once they finally realize the blazingly obvious! For today’s example let’s look at the big Dropbox data privacy controversy. There are a few serious problems with Dropbox, such as not requiring a password after a host is added, making it super easy for someone to pretend to be you (if they get your host ID) and access your data. That’s not great, but there are far worse things out there I worry about. But the big controversy is that… ghasp… Dropbox employees could access your data! But if you know anything about security you know that if you get a nice, pretty web interface; then somewhere, somehow, the odds are an admin at the service provider can access your data. There are techniques around this using creative programming, but one look at the Dropbox code in your browser makes it clear they aren’t using anything like that. This is because the Dropbox web servers need to see your data to show you the web interface. Ergo, the servers can decrypt your data. Ergo, someone at Dropbox can see it. Now this doesn’t need to be true – they could have restricted the web UI to metadata and still encrypted file contents, then used a browser plugin (or maybe even JavaScript) to decrypt the files. But both options entail usability and security tradeoffs. A great example of how to manage issues like these is the CrashPlan backup service. CrashPlan offers a cascade of security options, each with usability tradeoffs, and all available to users. (All these options protect your symmetric encryption key, not the data itself): Protect with your account password. CrashPlan can access and see your data if needed. Protect with a separate data password stored locally. CrashPlan admins can’t access your data (even to restore it). You need to keep and secure an extra password. Set your own encryption key. Can be on a per-machine basis. Very secure, requiring more management. There is, of course, much more to their encryption scheme – this is just the user-controllable portion. Dropbox could do something similar: Standard (perhaps the only option on their free plan): Basic account username/password as they have now. Enhanced Security: Set a personal password, with metadata in the clear. You can manipulate your files, but they can only be downloaded by the local agent (not via a browser) and you need to remember the password (no password restore capability). You can still share public files, which are stored in a separate directory using your account password as on the old system. High Security: Metadata and file data encrypted using your personal passphrase, separate from your account passphrase. Web UI can only manage public files – everything else is accessible only through their client. These would require serious development effort, and I don’t want to gloss over the complexity or importance of implementing this type of security correctly and safely. This stuff is hard. But it would be manageable if they made it a priority. But seriously, people – if you want something free/cheap with a pretty web interface to manage your data, odds are you are trading off security. I use Dropbox extensively and just encrypt the things I consider too private to expose. Share:

Oracle announced the April 2011 CPU this week, with just a few moderate security issues for the database. Most DBAs monitor Oracle’s Critical Patch Updates (CPU) and are already familiar with the Common Vulnerability Scoring System (CVSS). For those of you who are not, it’s a method of calculating the relative risk of software and hardware vulnerabilities, resulting in a score that describes the potential severity of the vulnerability if an attacker were to exploit the problem. The scores are provided to help IT and operations teams decide what to patch and when. Vendors are cagey about providing vulnerability information – under the belief that any information helps attackers create exploits – so CVSS is a compromise to help customers without overly helping adversaries. Oracle uses CVSS scoring to categorize vulnerabilities, and publishes the scores with the quarterly release of their CPUs. When Oracle database vulnerabilities are found, they provide the raw data fed into the scoring system to generate the score included with the patch announcement. Most of the DBA community is not happy with the CVSS system, as it provides too little information to make informed decisions. The scoring methodology of assembling ‘base metrics’ with time and environmental variables is regarded as fuzzy logic, intended to obfuscate the truth more than to help DBAs understand risk. The general consensus is that risk scores have low value, but anything with a high score warrants further investigation. Google and 3rd party researchers become catalysts for patching decisions. Still, it’s better than nothing, and most DBAs are simply too busy to make much fuss about it, so there is little more that quiet grumbling in the community. Things seem a bit different with the April 2011 CPU. One of the bugs (CVE-2010-0903) was very similar in nature and exploit method to the last Oracle patch release (CVE-2011-0806), but had a dramatically lower risk score. The modification to the CVSS security score was based on Oracle’s modification to the CVSS scoring system to include a ‘Partial+’ impact metric. I have not spoken to anyone at Oracle about this, so maybe they have a threat model that demonstrates an attacker cannot get out of the compromised database, but I doubt it. It looks like an attempt to “game the system” by producing lower risk scores. Why do I say that? Because a ‘Partial’ reference makes sense if the scope of a vulnerability is localized to a very small part of the database. If it’s the entire database – which is what ‘Partial+’ indicates – pwnage is complete. Lowering of CVSS scores by saying the compromise is ‘Partial+’, instead of ‘Complete’ deliberately(?) misunderstands the way attackers work. Once they get a foot in the door they will automatically start looking for what to attack next. To reduce the risk score you would need to understand what else would be exposed by exploiting this vulnerability. Most people in IT – if they do a threat analysis at all – do it from the perspective of before the exploit. Few fully consider the scope of potential damage if the database were compromised and used against you. I can’t see how ‘Partial+’ makes things better or provides more accurate reporting, but it’s certainly possible the Oracle team has some rationale for the change I have not thought of. To me, though Partial+ means a database is an attacker platform for launching new attacks. And if you have been following any of the breach reports lately, you know most involve a chain of vulnerabilities and weaknesses strung together. Does this change make sense to you? Share: