Ears always tingle in the decision analytics community when a mainstream publication talks about the things we do, and last Friday, the stimulus was provided by this article in the Wall Street Journal on Monte Carlo. Usually I reluctantly read these pieces fully expecting to cringe at the reinforcement of misconceptions and/or zombie fallacies, but in this instance I have to admit I was positively surprised. The author, Jo Craven McGinty, takes the reader on a broad tour of the applications of Monte Carlo simulation broadly defined (i.e., pretty much any probabilistic method that doesn’t involve a closed form solution, including anything you would do in with DPL or decision trees in general). The article notes the limits of the method (“garbage in, garbage out”, aka GIGO), but steers clear of naive suggestions that the quality of a forecast is based solely on whether it “comes true” in a particular case. Speaking to probabilistic evangelist Sam Savage probably helped the piece.
If one were to quibble (and really, how could one resist?), a point I find conspicuously missing is that all of these forecasts exist to help people make decisions. We want to know where a hurricane might make landfall because we have to decide if we’re going to evacuate a city. Complaints about the GIGO law are all well and good, but somebody has to make a decision. Do we wish we had more information, more precise measurements of ocean surface temperatures, market shares, efficacy, wellhead pressure, etc? Of course we do, but somebody has to decide, even if the decision is to refrain from taking action. Expert assumptions can be flawed, past performance is not indicative of future results, all health warnings apply, but a well-crafted forecast taking uncertainty into account is always going to provide more insight than shrug and a point estimate. One of the reasons we tend to favor decision trees over other constructs is that by explicitly including decisions in the model that produces the forecast they keep the focus where it belongs.
The corollary to the GIGO law is YHTMAD: You Have To Make A Decision.