Choosing Ranges for Tornado Diagrams
Tornado diagrams are useful tools for determining which variables in
a model are sensitive. These sensitive variables should be modelled as uncertainties
while the other variables can be left as value nodes.
Because we are comparing variables, we want to use a rule for selecting ranges
that ensures we compare apples to apples. Without such a rule, it can be very
easy to bias the results — if we put in a wide enough range, we can make
any variable move to the top of the tornado!
The simplest rule, to vary each number by the same amount, is clearly silly.
Some numbers in the model may be in millions of dollars, others in units of
production, and still others in years. No constant amount, such as 100, makes
sense in all of these cases.
A more reasonable rule might be to vary each number by a fixed percentage.
The benefits of this method are that it requires very little effort to implement
and is easily explained in presentations. While this method is tempting, it is
very rare that all values in a model will be equally uncertain in percentage
terms. In fact, with such a method you can get multiple answers for the same
variable by choosing different units: should we calculate +10% of 2007 sales or
+10% of sales growth between 2006 and 2007? A tornado diagram generated by a
fixed rule is often misleading, and can result in bad modeling choices.
The best rule for choosing ranges is to capture the same amount of uncertainty.
Commonly, this means a range that represents about 80% of total uncertainty
for each variable. This is often called the "10-50-90" approach because
it uses the 10th, 50th and 90th percentile points on the cumulative probability
distribution.
Why does this matter? Consider the following example. Suppose we own a paper
clip manufacturing plant that another company has offered to purchase. The offer
consists of a fixed amount of cash plus stock options whose future value is
uncertain. Our DPL model (based on a 5-year forecast) might look like this:
Someone suggests running a tornado diagram using the fixed percentage methodology.
We thus choose the high values for each variable as 10% above nominal and the
low as 10% below nominal. The result is the following tornado diagram:
According to this tornado diagram, Profit is most sensitive to the market price
of paper clips. Manufacturing costs and the price of steel are also sensitive
variables, while Profit is not very sensitive to sales and completely insensitive
to the sell-off price.
If instead experts are consult and data is gathered that covers 80% of the uncertainty of
each variable, then different values for the highs and lows will be used. For
example, it turns out that paper clips are in a very mature market and thus
the uncertainty surrounding the market price is very low. When the collected data is input, the following
tornado diagram results:
The second tornado diagram is significantly different than the
first. Manufacturing Costs is the most sensitive variable, and more importantly,
both Sales and Selloff Price are decision-sensitive (indicated by the change in color
of the bars). Therefore, model all five variables as uncertainties.
This example illustrates that using the fixed percentage approach can yield
misleading results. This approach does not make logical sense as
some variables are naturally more certain than others. While the state of some variables
may be "known" to ±5%, there could well be others that have uncertainty
bands of ±100% or more. Thus, while running a tornado based on fixed percentage
deviation may be simple and easily explained to others, doing so is likely to
reduce the validity and accuracy of the model.
| 
