Mid-Project Update and Trip Report
The week before last I headed up to Microsoft to present a mid-term report on Project Quant.
I was up there for the better part of two days, and mostly gave the attached presentation to internal teams so they had an idea of what we've been working on. It also allowed me to get some face-to-face feedback on the project. As much as I'm a virtual kind of guy, every now and then it's good to throw your ideas in front of a live audience.
In keeping with out transparent research process, here's a quick summary of some of the feedback I received:
- The model doesn't currently account for predictability or maintenance windows. (While Microsoft obviously is a bit invested in patch predictability, I don't consider this an MS-specific bias). I'm struggling to figure out how to incorporate this into the model with "hard" metrics, since downtime or disruption isn't always the easiest thing to measure until it happens. I suspect that's how we'll have to include it in the model, but am open to better ideas.
- The model doesn't currently account for extended-support programs where an end-user requests a specific patch from their vendor. A variety of vendors offer this for post-support and/or specialized products, and it's very common even in mid-sized organizations (depending on vertical).
- The model doesn't incorporate feedback to the vendor on patches -- e.g., for bad patches or improper documentation.
- There are other organizations we should look at contacting to get more community involvement (I lost my copy of the list, so can't post it here -- one was the Security Leader's LinkedIn group, which I am a member of -- I sent them a message to last week).
- The initial survey results are very interesting -- especially since it looks like some of the weakest areas of patching are aligned with the highest growth areas of attack (e.g., desktop applications).
- The model doesn't account for the costs of not patching. I think we will have to be very clear that that isn't the purpose of this model, and additional risk analysis is better suited for those costs. On the other hand, that opens up the model to be misused to state that patching provides no benefit. I'm open to ideas on this one, so long as they are hard metrics we can really measure. Perhaps I'm just not being creative enough.
Those were the major points (I wasn't keeping notes). Most of the discussion was around the survey results and the top level structure of the model.
Since that visit, we are at 76 survey responses and I've posted all of the detailed phases of the patch management process.
So if you are curious to see what I presented, take a look:
And don't forget to fill out the Open Patch Management Survey!
—Rich
