Why Experts Suck At Predicting The Future

Society loves experts! Why? Because people are highly persuaded by authority; and being an expert is considered being an authority. But experts are human, and therefore are fallible in their judgment; especially when it comes to predicting the future.

In certain domains, where things are very predictable and have short-time horizons, experts are very useful; but not in domains where the opposite is true. Experts give answers, yet the future is rarely created with the same answers; but with new questions.

I’ve written extensively about the difference between experts and Generalists, and how specialists are optimizers. So I was delighted to see that David Epstein, author of The Sports Gene, has written a new book about Generalists; Range: Why Generalists Triumph in a Specialized World.

One topic he looked at is futurism, and how experts are worse at predicting the future than generalists. The reason experts suck at predicting the future is because they approach forecasting with a narrow lens and are biased against any information that contradicts their beliefs:

The highly specialized hedgehogs knew “one big thing,” while the integrator foxes knew “many little things.”

Hedgehogs are deeply and tightly focused. Some have spent their career studying one problem. Like Ehrlich and Simon, they fashion tidy theories of how the world works based on observations through the single lens of their specialty. Foxes, meanwhile, “draw from an eclectic array of traditions, and accept ambiguity and contradiction,” Tetlock wrote. Where hedgehogs represent narrowness, foxes embody breadth.

Incredibly, the hedgehogs performed especially poorly on long-term predictions within their specialty. They got worse as they accumulated experience and credentials in their field. The more information they had to work with, the more easily they could fit any story into their worldview.

And the reason Generalists are better at making predictions is:

Tetlock, along with his wife and collaborator, the psychologist Barbara Mellers, ran a team named the Good Judgment Project. Rather than recruit decorated experts, they issued an open call for volunteers. After a simple screening, they invited 3,200 people to start forecasting. Among those, they identified a small group of the foxiest forecasters—bright people with extremely wide-ranging interests and unusually expansive reading habits, but no particular relevant background—and weighted team forecasts toward their predictions. They destroyed the competition.

Tetlock and Mellers found that not only were the best forecasters foxy as individuals, but they tended to have qualities that made them particularly effective collaborators. They were “curious about, well, really everything,” as one of the top forecasters told me. They crossed disciplines, and viewed their teammates as sources for learning, rather than peers to be convinced. When those foxes were later grouped into much smaller teams—12 members each—they became even more accurate. They outperformed—by a lot—a group of experienced intelligence analysts with access to classified data.

So, what’s the difference?

In Tetlock’s 20-year study, both the broad foxes and the narrow hedgehogs were quick to let a successful prediction reinforce their beliefs. But when an outcome took them by surprise, foxes were much more likely to adjust their ideas. Hedgehogs barely budged. Some made authoritative predictions that turned out to be wildly wrong—then updated their theories in the wrong direction. They became even more convinced of the original beliefs that had led them astray. The best forecasters, by contrast, view their own ideas as hypotheses in need of testing. If they make a bet and lose, they embrace the logic of a loss just as they would the reinforcement of a win. This is called, in a word, learning.

All of this isn’t to say that Generalists are perfect forecasters, no one is. But rather that they’re more competent at it because they approach it with a wider perspective. As Phillip Tetlock wrote in his books Super Forecasters, Generalists are more open to being wrong and that makes them better at forecasting.

Remeber, the future is not a set point, it’s a range of possible outcomes; Generalists are great imagining a range of possible outcomes.

Bottom line: Experts are blinded by their own expertise.