Building Long-lived Decision Models

Are you building a house, or just pitching a tent?

When you read about decision analysis models in case studies, they're usually described in the narrative as part of an epic process, one that begins with organizational angst and ends with a resounding consensus to pull the lever on that "irrevocable allocation of resources". Right, and sometimes that really happens! But more often, the allocation of those resources is gradual, and even when a single large chunk of capital has been committed, there are steps along the path where the project is reevaluated. If the probability distributions on the drivers of value have shifted, it may not make sense to continue. Maybe there's suddenly a better treatment option for the disease your drug is targeting. Perhaps you find yourself trying to import LNG into North America in the face of a fracking boom. There are very few truly fire and forget decisions; most of the time there is an opportunity (and an obligation) to reconsider as the future plays out.

In some cases, those options are modelled explicitly (and valued) in the decision tree as downstream decisions, but even when they are not, the value of the project can only be greater because you have them. Being ready to react and adapt requires a different organizational skill set and places more requirements on the model than would be needed for a one-off. A big decision can mobilize a team, and often you have the luxury of getting all the key people together around a table. But when it comes time to do a refresh, perhaps 12-24 months later, you probably won't have the resources to do that again. In a global organization, those face to face meetings require a lot of jet fuel. Most of the time you'll need to make due with virtual substitutes: teleconferencing, web meetings, or just emailed spreadsheets.

If you're building a house, and you expect it to last a long time, and you plan for maintenance, repairs and updates. In that spirit, these are some tips to keep in mind when building a decision model that will withstand the test of time:

  • Give the model a thorough review. Yes, it sucks to find a mistake a week after that big presentation, but if you wait for months or years, you may not even be able to deduce the intent, and a simple error may look like a forgotten assumption, which brings us to...
  • No magic tricks, inside jokes, or other modelling sleight of hand. When modelling under time pressure, sometimes the most expedient way of accomplishing something is a cute trick that another modeller (or yourself a year hence) could never be expected to figure out. Do what you need to do, but by all means clean it up and/or document it profusely before filing away the model for future use.
  • Observe the separation of data and structure. Never mix numeric assumptions and calculations in Excel formulas or DPL node data. In Excel, group the assumptions on one or a few sheets. In DPL, consider using init links so non-structural model updates will require only spreadsheet changes.
  • If the information experts are in a remote location you can only occasionally visit, try to leave a trail of methodological knowledge. Is there a wonky/analytic finance type on site who can be deputized for probability assessment updates? We all like to believe we're indispensable, but a face to face meeting with a well-coached novice may be preferable to phone call with a distant guru.
  • Know (and if possible, document) the trigger events that would require more than an update. Is it time for a teardown? Sometimes the world has changed so much that a reframing is appropriate, and if all you're doing is revisiting a few 10-90's, your decision quality is going downhill fast.

The model is dead. Long live the model!