No One Grades the “Experts”

The last note argued that the noise is structural. Here is the question that follows. When someone goes on air and tells you what happens next, and they turn out to be wrong, what happens to them?

Nothing. They are back on Thursday. No score is kept, no record is published, and nobody watching is expected to remember. A prediction is treated as content rather than as a claim, and content expires.

Somebody did keep score once. Between 1998 and 2012, the CXO Advisory Group graded 6,582 public stock-market forecasts from 68 well-known forecasters against what the S&P 500 went on to do. The average forecaster was right 47 percent of the time, and 42 of the 68 scored below a coin flip. The biggest macro call fares no better: across 63 countries over two decades, consensus forecasts saw only 5 of 153 recessions coming a year ahead.

There is no mystery about why that scoreboard never made it on air. A stage that kept score would get hedged answers, and hedged answers do not hold an audience. It depends is true most of the time and unwatchable all of the time. So the stage selects for people who sound sure. Nobody has to lie for that to happen.

That is worth knowing, because confidence is what you are really reacting to. When the grade never makes the broadcast, sounding certain costs the speaker nothing at all. It is the cheapest thing on the screen and the easiest to hear.

References

CXO Advisory Group (2013). Guru Grades. cxoadvisory.com/gurus. An, Z., Jalles, J. T., & Loungani, P. (2018). How Well Do Economists Forecast Recessions? IMF Working Paper WP/18/39.