Please Don't Use Words to Define Probabilities

In this space we often write about the do’s and don’ts of probability assessment and working with uncertainty in general. One of the most fundamental don’ts is that you should not use words to define probabilities. You should more than avoid it, you should never do it. A 15% chance of something happening is exactly that: a 15% chance. Write it as 0.15 if you like, but don’t call it a “moderate risk” or say it “probably won’t happen”. Once you allow words to serve as a substitute for unambiguous quantitative information you’ve transported your analysis back several centuries to the Dark Ages, before there was any math behind the natural sciences. Worse, you might at some point want to build a model, and then you’ll make up a dubious little table to translate those words into numbers, and your table will have nothing to do with the one in the mind of the expert who assessed the risk in the first place.

In decision analysis training workshops, we often perform an exercise where we ask people (working individually) to translate uncertainty words into numbers. Even the simplest example, “It will probably occur”, can give rise to numbers anywhere between 50% and 90%, depending on the person. (I say 50% plus epsilon, but that’s just one man’s opinion.) Numbers about probabilities don’t have this problem. We all use the same math, so 75% means the same thing to me as it does to you. NWS RiskCategoriesTable

An excellent and colorful example of this truth comes to us from the (US) National Weather Service. I’m normally a big fan of the NWS: they provide a lot of useful information, and generally communicate it well, including the degree of uncetainty in the forecast. However, with their new thunderstorm risk categories they have really laid an egg. Take a look at the chart to the right.

I think we can all agree that High Risk sounds much worse than Slight Risk. But some of these levels are even directionally ambiguous: is Marginal Risk better or worse than Slight Risk? To me, Enhanced Risk sounds worse than Moderate Risk, but the chart says otherwise. If we need to have these categories, why not just say Level 1 through Level 5? Or better yet, give us the numbers: if there is a 30% chance of severe thunderstorms tomorrow, just say it. (That would be Enhanced Risk, by the way). The little table that takes you from words to probabilities does exist, but it’s not easy to find (try here). People who pay attention to this stuff are weather geeks, and I’ve never met a weather geek who was afraid of a clearly stated numerical probability.

There is usually someone who argues that the difficulty of subjective probability assessment – “Could be 20%, 30%, I don’t know” – makes the value of clear, quantitative information less important. But this argument defeats itself: if the probability assessment came from a mentally challenging synthesis of disparate information, why would we throw more noise on the result by transcribing it to English?

So, give thanks for the math we share, and let your probabilities express themselves naturally.

“Writing about music is like dancing about architecture”

–Martin Mull