5 Ways to Screw Up a Decision Analysis Project

(Or, how about just a little bit of process to go with that gorgeous decision tree?)

Personally, I've always loved a beautiful model, whether it's based on decision trees, Monte Carlo, dynamic programming, or any other quantitative method. I've made a few beautiful models myself, and have admired many more created by colleagues and DPL users. However, fairly early on in my career I began to notice that the correlation between the beauty of a model and the success of a project (as measured on the armchair decision quality index) was quite small. A good (or even great) model can't stand on its own.

I don't have a lot of patience with process. However, I very much want to a) get the right answer and b) make the decision maker(s) understand that answer (or what's the point?). Doing both requires a decent project plan and some soft skills.

Process and modelling are a kind of yin and yang of normative analytics. Without a good process, great quantitative modelling is a waste of time, no different than writing a book and throwing the manuscript on the fire. Just the same, dodgy business schools, life coaches and self help paperbacks sell thousands of shovels full of decision making advice which is at best useless since it has no grounding in management science. Decision analysis is powerful because it's practically doable and (in some sense, which we can rigorously define) actually correct. Feel good, let's-share-the-experience quasi team building business processes are cheap; DA is something more.

There's more than one way to do it right, and many authors have documented their preferred approaches, which usually involve between 3 and 7 steps. But it's much more fun to talk about ways to do it wrong. Below are five good (bad) process errors that can screw up a project. Do two or more out of the five and you are guaranteed a decision disaster.

  • Blow Off Framing
    Framing can be tedious, especially with austerity-mode clients whose meeting rooms are never adequately catered. Skip it if you can. They pretty much knew what they wanted when they hired you, right?
  • Any Team Will Do
    Mustering a congenial, productive project team is easy. Take a "coalition of the willing" approach. Generally, people whose work schedules allow them to take long lunches will be the best team members.
  • Embrace Senseless Complexity
    If possible, utilize a large, inflexible, highly-detailed deterministic model, and let its constraints guide your modelling. Working meetings can focus on feeding the beast.
  • Just Give Me a Number
    Do probability assessments via email. Just send out a questionnaire. For range assessments, make sure the base case number is provided and formatted in bold type.
  • Build Suspense
    Don't meet with the actual decision maker(s) until you're presenting results. Who knows, maybe you answered the right question. It's more fun to be surprised.

We do Decision Analysis because we want to get the right answer, and we should never lose sight of that in our pursuit of shared understandings, buy-in, and other fuzzy stuff that we know we also need.