Possibility is not Probability

Excellent, Rich.  I’d add this: Security specialists and their partners need to have skills to deal with *both* the space of possibilities and also probability estimation (on the way to risk estimation and management).

We need to constantly explore and monitor the space of possibilities to minimize unknown-unknowns and to rule out certain threats as improbable.  It is also essential for agility, i.e. preparing for emerging threats, or at least being in a position to adjust if they become probable.

I would be willing to bet that the vast majority of security professionals have little or no training in this stuff, beyond basic probability.

I’m starting to like you more and more every day.

I agree completely. It also ties in with increasing a focus on incident response, rather than completely on prevention.

Here’s one more example of how the probability of events are poorly managed.  Consider how many organizations impose draconian security controls to avoid some highly improbable threats, but those very controls make it *highly probable* that users, managers, and administrators will circumvent the controls because they find them too burdensome.

“I would be willing to bet that the vast majority of security professionals have little or no training in this stuff, beyond basic probability.”

I think you are being overly generous on how much most of us know about basic probability. :-)

For the record, you’ve completely misconstrued my argument from Friday, which had absolutely nothing to do with either “possibility” OR “probability.” Really classy, Rich.

You make a large number of errors in this post…

1) You should really say “Possibility is not certainty.” This whole “possibility is not probability” phrase is pure nonsense because at their root they all deal with chance. Relying on colloquialisms to make your point is folly here.

2) “...the odds of an average Mac user being infected by any type of malware are so low as to be unmeasurable…” - You’ve yet to provide a basis for this assertion. I maintain that so few people are looking for Mac malware that this is a self-fulfilling prophecy of the worst kind (because it’s based in blind ignorance rather than informed study). If you’re so sure this is true, then you will do us all a favor and thoroughly document your assertion. Until then, I call foul.

3) “Throughout the security industry we continually burn our intellectual capital by emphasizing low-probability events.” - This is apparently the Securosis theme for the month? Blame infosec professionals for the failings of vendors and the FUD of vendor marketing? This is patently insulting tripe that has no business being directed at those of us in the field trying to get stuff done. It’s also patently false for practitioners. I don’t know anybody serious in this industry (except maybe Anton) who thinks FUD-based arguments looking at low-probability events is useful or important. On the flip side, it is sheer lunacy in certain planning cycles (e.g. BCP/DRP) to ignore high-impact low-frequency events like natural disasters, so be careful how you phrase it.

4) “...we can’t predict that with any certainty and until then our response should match the actual (current) risk.” - What’s your point here? You’ve made an arbitrary “risk” statement without contextualization, and thus it’s absolutely meaningless. Quit talking about “risk” as a general term because it’s absolutely meaningless without proper contextualization in a given business environment. You’re guilty of the use of risk as FUD, which is ironic given the thread on Friday.

5) “Possible isn’t probable. The mere possibility of something is rarely a good reason to make a security investment.” - And here we revisit the basis of your thesis: semantic games. Every security decision is based off possibilities, and to suggest otherwise is ignorant. If there’s no chance that something will happen, then why would we ever invest resources into it? The problem, again, is that you’re using these terms in colloquial ways, not in a way that is meaningful, especially from a statistical or scientific perspective. Go read the definitions, I think you’d be surprised just how ignorant and wrong your use is here.

Another fundamental issue is that we suck at quantifying how any given security measure impacts the probability of a vulnerability being exploited. 

So, we can’t accurately predict if a thing will happen thus we don’t know if it is worth acting against, and when we do act against it, we can’t accurately predict if the action will diminish the risk or by how much if so. 

Funny… my MS is in CompSci, but I’ve yet to see the science and it is continually depressing.

@Ben
>>“This whole “possibility is not probability” phrase is pure nonsense because at their root they all deal with chance. Relying on colloquialisms to make your point is folly here.”

I think you are mistaken.  There is a well developed philosophical literature on the distinction between possibility and probability, and also their relation.  “Possibility” is part of modal logic, which is reasoning about “necessity”, “possibility”, “actuality”, etc.  For a quick overview, see the Stanford Encyclopedia of Philosophy: http://plato.stanford.edu/entries/logic-modal/ and http://plato.stanford.edu/entries/possible-objects/ . For a thorough treatment that relates the two, see: “Reasoning About Uncertainty” by Joseph Y. Halpern.

For something to be possible, the logical prerequisites for it must be actual.  E.g. for macro objects to be possible, their prerequisites must first exist (atoms + forces to hold atoms together).

It’s a truism that you can’t estimate the probability of some event if you cannot first establish it’s possibility.  Furthermore, many probability methods depend on you ability to enumerate *all* of the possibilities (“mutually exclusive and collectively exhaustive”).  You don’t get there by probability analysis alone.

>> “On the flip side, it is sheer lunacy in certain planning cycles (e.g. BCP/DRP) to ignore high-impact low-frequency events like natural disasters, so be careful how you phrase it.”

Yes, yes!  In addition to having the skills and capability to estimate risk, we need to know when and how to use that information.  Any decisions that have a long time-horizon must include estimates of high impact/low frequency events.

Ben,

My understanding of your position from the debate on Friday is that we do not have the data to say that Mac users experience near zero levels of malware, and Mac users should thus take precautions like AV.

You’ll notice I left your name off the quote above since it was a combination of a few responses, only one of which was yours. You obviously took it personally, which wasn’t the intention and the reason for not naming you. Read the other @replies and my fake quote makes absolute sense.

Don’t think it’s all about you- it isn’t.

On to responding to your specific concerns:

1) No, I said exactly what I mean. The point is not folly, and is a common problem in both our industry and risk assessment in general. I meant that possibility is not probability, just as probability is not certainty. There is no logic flaw in that statement, using accepted definitions of possible, probable, and certain.

2) Please review the Symantec, McAfee, and various other malware reports. Look for two trends- rise of detected Mac malware, and rise of infected Macs. I don’t have time to pull all the reports, and I know you know where to find them. Your statement is based on ignorance, mine is informed. There is anti-malware on many Macs, cloud based filtering, and detection of source systems for attacks. All of these back my statement. No, I’m not showing the data here, but I just don’t have time to pull it and it’s well known and widely available.

My assertion is well backed by multiple sources of evidence which are publicly available and easy to find. We are talking about malware vendors admitting there’s little for them to fix, so I consider it more reliable than these reports often are.

3) Yes, this is a problem with our industry and a mistake I’ve made in my past. We aren’t perfect, and there are numerous examples. I do agree the vendors take much of the burden, but how about those PKI deployments pushed by practitioners?

You may not have played this game (which I doubt), but many in our industry at the practitioner level have and do. It’s the nature of working in a risk-based profession, and human behavior.

4) Don’t blame me for your inability to understand the context. Risk is the loss or potential for loss (the RMI definition). The current risk is the potential for loss, which is a combination of the chances of the event and the potential severity. Since my post focuses on probabilities, I assumed the reader would understand that I’m referring specifically to the chances of a user encountering and being infected by malware.

Seems clear to me. Plenty of context since that’s what I spend the entire post talking about.

5) I can’t see you making a point here- other than trying to make this personal. This is a nothing but a straw man, since it does not represent the content of the post nor my position.

Now perhaps you are offended in that I used to term probable to equate that something is likely to happen (odds over 50%) as opposed to the the scientific definition where a probability is the specific measurement of exactly how likely something is (the exact percentage, or other scale of measurement).

Don’t assume that is a lack of knowledge or an error. Using the colloquial in an informal blog post doesn’t make the conclusion incorrect; this isn’t a scientific paper and so far you are the only one who is concerned with the usage.

>> “On the flip side, it is sheer lunacy in certain planning cycles (e.g. BCP/DRP) to ignore high-impact low-frequency events like natural disasters, so be careful how you phrase it.”

As someone who responded to Katrina, among many other low-probability events involving life threats, this is a lesson I know all too well.

And this is a reminder that heavy commenters can register for the site and I can set it so you don’t go through moderation. I’m trying my best, but hard to keep up on a thread like this.

@Russell - Contrast this against common language dictionaries. You will find that “possibility” is defined as a synonym for “likelihood” which is defined as “probability.” This really just underscores my point that the phrase is a bad choice and doesn’t make any sense. Saying “possibility is not certainty” would make a whole lot more sense.

@Rich -

On #1… the definitions are not consistent, it depends on where you’re reading things from… look at mainstream dictionaries and you will find that possibility is being defined as a synonym for likelihood, etc. It’s not a good choice of words. Same goes for your use of probability… I’ve been suspecting that you meant it in the sense of “greater than 50% chance,” but it’s been completely unclear… once again, choice of words - and clear definition - is extremely important…

On #3… why does everybody hate on PKI? it’s not like it just fell off the turnip wagon… the problem was in thinking that every enterprise needed one… I lived through that period, in the field, and I honestly still don’t know where that came from… as best as I can tell, the vendors were again the source…

On #4… no no no no no… this is your favorite definition for “risk” but that is NOT the same as defining risk in a context. Everybody seems to get this wrong. Your definition is only textbook - what does it mean applied? Theory in this case is only as good as it’s application, and “risk” is completely screwed up. You CANNOT go around saying definitively what is and is not a high or low risk for everybody, because every organization is different, their requirements are different, their priorities are different, etc. This is a MAJOR failure in the industry today. Everybody flogs “risk” like it’s some useful generic term, and it amounts to nothing more than FUD. Call me if you want to discuss, because this is a very, very, very important point.

On #5… every security decision is based off of the possibility that something bad could happen… do you not agree? or do you think that decisions are made for arbitrary reasons without any attempt to pin it to something? even FUD is based in the possibility that something might happen, though often exaggerated beyond the reasonable likelihood of the outcome. That was my point, and I think you missed it, too. This point, incidentally, is not ad hominem… if you want that, read my ranty post from today (or don’t, it’s not worth the paper it’s printed on)...

As for registering to skip moderation, I did, and it isn’t…

@Ben I can certainly understand how common and informal definitions can get in the way of their use in formal analysis. (The word “security” is a fine example.)  However, that shouldn’t be an insurmountable obstacle.  If we are doing formal analysis, we should draw on terminology that has been defined in the context of that formal analysis, as I pointed out in my references.

BTW, I took up your challenge and looked up “possibility” in Wikictionary:  http://en.wiktionary.org/wiki/possibility .  There is no mention of “likelihood” or “probability”, even in the list of Synonyms or Related Terms.

Ben and Russell- found your registrations and set you for non moderation.

Ben- after your blog post, which was nothing than a very long insult, I won’t try and engage you in debate again on this (or probably any) issue. You can continue to post here, as long as it isn’t purely inflammatory.

@Russell - check out dictionary.com definitions, too, which aggregates several sites. My point is simply that there’s inconsistency, that Rich used the terms informally, and now we’re talking about formal definitions, and it’s confusing as heck. Ergo, the point doesn’t carry very well.

@Rich - Yep, the post was absolutely and positively a vitriolic rant against you. It was, in fact, the very embodiment of an ad hominem argument, despite the interspersing of other arguments that I’m sure will be lost by anybody who wastes their time reading it. I found your interaction on Twitter Friday, along with this post Monday, to be that offensive, both personally and professionally. It is precisely your haughty tone throughout this entire episode that underscores why many of us in the industry find analysts so infuriating.

No harm, no foul. Rich your point is right on (how is that for captain obvious?).  That is the whole point of risk management. Managing the risk of the probable versus the possible, at what cost.  It is also related to a recent court case throwing out a data breach damages suit. Just because there was a breach and there are theoretical damages is not enough. You need real damages. No harm, no foul! I have written more about it on my blog at http://www.ashimmy.com/2009/12/no-harm-no-foul.html

Ben,  Do you wear a helmet, five-point harness and a fire suit when you are driving around town?  It’s possible that your seat belt and airbag aren’t enough.

@rmogull (Rich) – There is a part of me that is thankful for being late to the game on this post. Here are my thoughts on a possibility and probability. If we can agree at a high level that risk is the frequency and magnitude of loss – then this is a simple problem. Frequency best aligns with probability – not possibility. It is possible that that the sun could stop emitting light in the next 36 hours – but it is highly improbable. I think your post hits the mark.

One of the best ways I have had the “probability vs. possibility” concept explained to me is from Jack Jones – the founder of the FAIR methodology. Imagine you have a revolver with one bullet in the cylinder and a semi-automatic pistol with a full magazine – one round in the chamber. In a game of Russian roulette – we know that is possible that death could occur with either weapon. However, if given the choice between the two weapons – most reasonable people would choose the revolver because the chances of death are 1-in-6 versus the semi-auto where the chances of death are pretty much 100%.

The same concept applies for information security risks. We all know there are super elite hackers out there or other threats that we cannot 100% protect against. General speaking, we cannot manage infosec and related investments towards what is possible but what is probable (I think government intelligence agencies are the exception to this).  If we did, we would be over-spending in IT security, erode company profits, lose credibility and ultimately find ourselves out of jobs.

I get the point of your title, Rich, but I think it’s unfortunate to make it under the often-impassioned debate that is Mac malware.

Even despite numbers, there is still debate on what is probable. Some may find it far more probable that Mac malware will occur while others don’t think it is probable while others will say, “If it’s not here, it’s not probable at the present and I’ll worry about it tomorrow only after it actually happens.” Some simply believe you should just run AV, as a matter of best practice, regardless. It is, sadly, a passionate debate.

It might be another bad example (human life is always impassioned), but with Hurricane Katrina wasn’t that an unlikely event and a decision based on some scale between probable and cost? <—feel free to leave that as rhetorical. I try not to dive into that topic too much, as many people know far more about it than I do.

This really reminds me also of Schneier’s (or others, I’m not sure) remarks on risk and how we worry so much about improbable but possible events.