In a previous post, Chris expounded on the methods for generating good tornado diagrams. It’s relatively straightforward to create a meaningful tornado for a business problem defined by a few symmetric quantitative chance events (i.e., the kind with 10/50/90 assessments and Low/Nominal/High outcomes). But for analysts in the pharmaceutical industry evaluating early stage drug development projects, it’s rarely that easy, and producing an insightful tornado can be a real challenge.
Early-stage pharma drug development models (like the one shown above) are typically highly asymmetric and include a mixed bag of qualitative Success/Fail events and quantitative on-market uncertainties – the combination of which can result in some pretty meaningless tornadoes if dealt with carelessly. Yet, valuable insights can still be gleaned if one understands the analysis behind the tornado and, consequently, how one can manipulate events to answer critical questions. For example, management might like to know:
- What are the relative effects of the main commercial value drivers (assuming we make it to market)?
- How much have we lost if we fail at a given stage gate?
- How much more valuable would the project be if we could eliminate risk?
For an early-stage pharma R&D decision analytic model, how do you generate a…
..meaningless Default Base Case Tornado?
The naive (or lazy) analyst might simply click the base case tornado button, proceed with the defaults and generate the tornado diagram to the right – one that is confusing and tells you nothing.
Now that we have that out of the way.. I suggest you read on and become enlightened in the ways of meaningful pharma sensitivity analyses.
..*"Commercial Drivers" Tornado*?
Use branch control to control all success/fail events to success to effectively generate a “given success” tornado. Run a DPL Base Case Tornado with default settings in order to produce a diagram that informs your answer to the first question above: What are the effects of the main commercial value drivers given success?
..*"Impacts of Failure" Tornado*?
Use branch control to control all on-market uncertainties to their base case outcome. Run a DPL Base Case Tornado with the default settings. The tornado generated will help you to visualize just how much it will hurt (i.e., how much $ lost) if you fail in each of the 4 phases.
..*"Value of Eliminating Risk" Tornado*?
Use branch control to control all decisions to proceed and market uncertainties to their base case outcome. Then you’ll want to set the probabilities of success at each phase to 1.0. Lastly, create 4 value nodes to represent the original probabilities of success (e.g., Probability Phase 1, Probability Phase 2, etc.). Run a Value tornado on the 4 values in order to see how much more valuable the project would be if you could eliminate risk at each phase.
If you'd like a closer look here is the DPL model and linked spreadsheet: