The Securosis Nexus

To prioritize sounds good, how does that work in real life?
Take the Microsoft patches, which most of us have to deal with on monthly basis.
The MS patches use “Maximum Severity Rating” and “Exploitability Index”, what should one use to prioritize?
Should one only look at the criteria given by the vendors?
So it’s exploitable, maybe a worm could propagate etc. but you have efficient protection on the endpoints should you take your time now or run to get the job done? The main issue here is that the decision process itself can be time consuming. Maybe you are better off with a fast but less perfect decision framework?

The other issue: I have identified some critical and some not so critical patches. How much time should I invest in testing? How useful are the tests in the lab? It seems that everything works fine in a two day test in the lab but not in real life.
And what do we do now with the patches that are not that critical? You postpone them for later. But “later” you get another pack of patches that need to be analysed and deployed. This way “later” never comes for the less critical patches. And if you delay them for later that means another maintenance window where applications might not be available to users due to restart.

Prioritization sounds nice, but what I hope this project will come up with is something that can be implemented in real life and not just a nice to read white paper.

In terms of what criteria to use, I’ve always recommended a mix of data sources- the vendor’s rating, any external ratings, our own evaluation, and what assets the patch matches to (this is obviously all security specific). Most organizations I talk with tend to use the vendor details more than the ratings (e.g. remotely exploitable). Then they match to their infrastructure and any mitigating factors (security controls, or how that asset is being used),

As for testing, our initial research and survey results show that it’s all over the map, and depends a lot on the kind of patch/systems involved. Some database patches are testing for months, while some desktop patches only take minutes to hours. I think when we correlate testing effort with cleanup/breakage, that will help people figure out what level of testing is best for them.

What this comes down to is that I don’t think this project will make specific testing/prioritization recommendations… the model is designed to give you the tools to measure and optimize your own process. I’m hoping we can do some benchmarking after we finish the model, since I think that will give us the answers you are looking for.

Keep in mind we are producing an exhaustive, comprehensive model that includes a lot of things people may need to skip in their particular environments. I intend to be very clear on that when we write it up.

Does that help? I wish we had the answers, but I don’t think we (or anyone) has the right data to make generic suggestions with any sound backing.