We spend a lot of time talking about security metrics over here, and I’ve been pretty critical of both overly-broad initiatives that don’t help people get their day to day jobs done, and “fluffy” models that try to put hard numbers on risks/threats and such. Well, it looks like it’s time for me to put up or shut up.

I’m pleased to announce our latest metrics project, which we’re currently calling Project Quant. (Yes we need a better name). We were approached by Jeff Jones at Microsoft to help build an independent model to measure the costs and effectiveness of patch management. This will be a hard metrics model, focused on measuring the operational processes associated with patch management. The goal is to provide IT organizations a tool they can use to measure how effective they are, and track that over time.

I’m excited about this project for two main reasons:

  1. We get to focus on hard, practical metrics people can use to improve operations.
  2. We are following a “radical” version of our Totally Transparent Research process to ensure objectivity.

We’ve set up a dedicated landing area for the project at http://securosis.com/projectquant where we will be posting all the materials. Here are the bits you might care about:

  1. We are soliciting as much participation in the project as possible- including competing vendors, end users of all sizes, consultants, whoever.
  2. The project has a deadline of late June, so this won’t drag out indefinitely. The first version may not be perfect, but come the end of June there will be a first version.
  3. We really need you to get involved. We’ll be asking for survey participants, reviewers, and just plain ‘ol grumpy commenters to keep us honest, and help produce a useful result.
  4. The results will be released under a Creative Commons license in an open format.

We have the first two posts up at the landing site. The first, Introducing Project Quant, provides an overview of the project and the research process. The second, Project Quant: Goals delves into the project goals in more detail.

This is a pretty huge project, even though it’s laser focused on one single operational area. Hopefully you like the idea, and are interested in participating.