The attitude needed to be a better forecaster…and innovator

SuperforecastingPeople who are good at forecasting, including innovators, are good at changing their minds; an uncommon attitude. Changing ones mind contradicts the conventional wisdom of relying on experts for right answers. Truthfully, experts have no place in predicting future events because when it comes to discovering and predicting the unknowns, experts are overrated.

How so?

A new book by Philip Tetlock, a psychologist at the Wharton School of the University of Pennsylvania, has just written a book revealing their secrets. It’s called Superforecasting: The Art and Science of Prediction, and when it comes to predicting the future:

The superforecasters, it’s important to note, were pretty much your everyday kind of folk, including housewives and factory workers; they had no specialized knowledge and had no access to specialized information. Even still, their answers were, on average, about 30 percent more accurate than the experts’ answers.

How can this be?

In an interview with Science of Us, Tetlock said that superforecasters’ skill comes down to one thing. Surprisingly, it isn’t the attributes you might guess, like numeracy or a high level of general intelligence. “One of the discoveries is how much hinges on a person’s attitude,” he said. Most of us — experts included — make decisions too quickly, and change our minds too slowly. Superforecasters, on the other hand, keep an open mind when forming opinions, seeking information from a wide variety of sources. (They’re the wide-ranging fox to the expert’s super-specific hedgehog, Tetlock said, using the old analogy.)

But they also are okay with being wrong, and are able to revisit and revise their prediction when new information comes to light. As the website for the Good Judgment Project — that’s what Tetlock called the overarching research project, which has involved more than 20,000 participants, explains, “belief updating” is a key component to the superforecaster’s skill.

Confirmation bias is the enemy of forecasting

Experts are limited by what they know, and anything that contradicts what they know will not be taken as a positive. What makes experts so stubborn? It’s called confirmation bias, the propensity to find information, opinions, evidence that confirm what we know.

Expertise, along with group-think, are the enemies of innovation. Both are always present and must be deliberately kept at bay, one way to combat them is by being open to new points of view and deliberately looking for evidence that contradicts our thinking.

Innovators, like Jeff Bezos, have strong opinions weakly held. Jeff Bezos is an innovator who changes his mind all the time, as shared in this key observation about people who are right a lot:

He said people who were right a lot of the time were people who often changed their minds. He doesn’t think consistency of thought is a particularly positive trait. It’s perfectly healthy — encouraged, even — to have an idea tomorrow that contradicted your idea today.

Bezos himself puts it this way about how Amazon thinks about the future: Be stubborn on vision, flexible on details.

Right on!

Bottom line: The goal of forecasting isn’t to be right all the time, nobody is; rather it’s to be less wrong over time. And that happens through relentless experimentation, being open to new points of view, new information, new ideas, contradictions, and challenges to their own way of thinking.

  • Pingback: The attitude needed to be a better forecaster.....()

  • Hi Jorge,
    Thanks for introducing me to Philip Tetlock! That’ll go straight onto the reading list.
    I work on enterprise analytics problems and the idea of superforecasting is going to be helpful to provide a counterbalance to the influence of analytics lobbyists on C-suite investment decisions.
    One of the traps senior leaders are falling into is the lure of ‘expert+analytics’ having breakthrough answers to problems senior leaders aren’t fully thinking through.
    You’ve mentioned confirmation bias and groupthink as enemies of good decisions: I’d add framing bias to the list. An issue I see all the time are issues framed by experts in such a way as to make their advice a foregone conclusion.
    So thank you on behalf of my customers and please keep the great articles coming!.

    • Hi Rohan,

      Cool! True, I find “framing bias” around too, and it’s unfortunate because it’s very hard to differentiate between good intentions and fact checking.

      What is your take on “teaching people” (everyone) to understand / interpret analytics? Is that something that is still siloed to a special few or is it something that can an enterprise wide activity?



      • Hi Jorge,
        I think about this a fair amount. I volunteer as the Chairman of an analytics users community here in NZ. I come from the business side and observe how the two groups interact.
        It’s helpful to think about ‘analytics’ by substituting the word ‘calculators’. I use this to defuse much of the hype from the topic and (I) give us a sense of evolution of the technology and (ii) what it means to what we set out to do.
        So, can we ‘teach people’ to understand/interpret calculators? Yes we can. Teaching is a form of learning, so can people learn to understand calculators? Yes we can.
        There are a lot of moving parts to take into account, but we just want to get started and learn our way forward. So we want a few basic thinking tools to avoid getting caught up in the dramatic scientific soundbites of Big Data and the global calculators industry.
        Because it is an industry. Important to remember. There is a lot of money being spent (often foolishly IMHO) and there are a lot of careers riding on the big bets being made. One thing I’ve observed with senior decision makers is that analytics gives an opportunity to defer decision making.
        Given that good analytics takes the easy answers out of the decision process (so that the only decisions that get to the top are the really, really, really hard ones) giving senior people a good reason not to decide is a bad thing indeed.
        So, what are these tools? Fortunately, they’re well known through history and can be learnt through practice. Here are my top three:
        1. Occam’s Razor
        2. Bayesian Reasoning (not the branch of statistics)
        3. Estimation of 90% Confidence Intervals
        For a bonus, read the life of Eratosthenes and how he estimated the circumference of the Earth by observation and geometry.

        • Hi Rohan,

          Very good. As I understand it, teach people how to be less stupid via decision making models before teaching them how to use and interpret analytics so they don’t fully trust that the tool will do all of the work for them.

          Makes sense.

          Have you written / talked about this?

          Thanks for sharing,


  • Pingback: Rohan Light On Forecasting And Data Analytics | Game-Changer()

  • Pingback: The attitude needed to be a better forecaster.....()