Project Quant: Test and Approve Phase

Last week I spent a couple days up at Redmond presenting our work-to-date on Project Quant, and it was great to get some face to face feedback. I'll be posting a trip report (probably tomorrow) with the presentation I used and the feedback we received. Although we haven't reached 100 responses yet, the preliminary survey results are very interesting and we included some highlights in the presentation. (Please don't forget to take the survey).

But first I want to push out the next detailed phase in the cycle: Test and Approve. It's been a long time since I've been responsible for testing patches myself (and where I worked we were pretty immature about it), so this one could use more review than some of the other sections.

From my thinking, the biggest factor for any given test will be the number of test cases/dependencies. As I put the model together I'm realizing that some of these numbers won't be predictable if you haven't already measured them, and may not apply to "new" cases. For example, you can't know how long a given test for a given asset will take until you try it, although I suspect we will see some sort of predictability emerge based on trends. Also, there's another sub-cycle as tests either fail or produce other unwanted results.

In other words, this model isn't fully predictive, and is more oriented towards measuring what you currently do and tracking that over time. Then I think some predictive elements will emerge once you have a good base of metrics to work off. That should help those of you who try to use the model to help optimize your processes.

Just some rough thoughts that are a little out of scope of the details of this particular phase, but I think it's important to keep the context in mind as we build it out so we don't drift off course.

—Rich