The Fallibility of Experts: How Experts Lead Us Astray and How to Prevent It
/Be careful what you ask experts. Experts are bad at predictions, but are great at assessing base rates.1
Experts are the go-to source for life’s uncertainties. We rely on them for guidance and advice during ambiguous environments. We crave their authority, confidence, and conviction.
But the evidence is clear - experts are awful at predicting the future. Expertise is not synonymous with superior forecasting ability.2
Experts are extremely useful, if properly directed. As we’ll see, we can use an expert’s deep knowledge to improve our decisions. However, the process is not intuitive and doesn’t happen automatically. The burden is on us to ask the right questions.
It’s especially relevant for investing. Investors rely on consultants, economists, and managers to predict the future and position portfolios accordingly. But because we ask the wrong questions, we engage experts at what they are bad at - prediction. We fall into the same trap, desperately trying to predict the future.
As stated in the beginning, experts are bad at prediction but great at assessing base rates. That’s the secret to using experts. By focusing on base rates, rather than predictions, we’ll effectively use experts to make better investment decisions.
Chip and Dan Heath, authors of Decisive: How to Make Better Choices in Life and Work, highlight the strengths and limitations of experts:
We use experts the wrong way. We lean on them to tell us about the future. Experts aren’t any good about predicting the future. What they are good at is explaining how similar situations have worked out in the past.3
It’s dependent on us to harness the power of experts. Here’s how to do it.
Prediction Isn’t the Goal
The evidence is clear – experts can’t forecast the future. As Nate Silver, author of The Signal and the Noise: Why So Many Predictions Fail--but Some Don't, states:
We need to stop and admit it: we have a prediction problem. We love to predict things—and we aren’t very good at it.4
Experts are bad across all fields, not just investing. Philip Tetlock, author of Superforecasting: The Art and Science of Prediction, highlights the fallibility of experts:
…The experts in his [Tetlock’s] survey—regardless of their occupation, experience, or subfield—had done barely any better than random chance, and they had done worse than even rudimentary statistical methods at predicting future political events. They were grossly overconfident and terrible at calculating probabilities…It didn’t matter whether the experts were making predictions about economics, domestic politics, or international affairs; their judgment was equally bad across the board.5
While experts have a poor track record for prediction, they still have information that can help us, but only if we correctly frame our questions.
The Right and Wrong Way to Use Experts
Experts are good at assessing historical context and thinking in terms of base rates. They are better at building probabilities than predicting the future. It seems counterintuitive, but an expert’s value comes from thinking about the past, rather than forecasting the future.
Incorporating base rates is also referred to as the “outside view”. Daniel Kahneman, Nobel Prize winner and respected economist and behavioral scientist, describes the unintuitive nature of the outside view:
Statisticians call that the base rate—how common something is within a broader class…He calls it the “outside view…The outside view is typically abstract, bare, and doesn’t lend itself so readily to storytelling. So even smart, accomplished people routinely fail to consider the outside view.6
In contrast to the outside view, the inside view overemphasizes the details of the current situation:
…the “inside view,” which is the specifics of the particular case… It’s natural to be drawn to the inside view. It’s usually concrete and filled with engaging detail we can use to craft a story about what’s going on.7
Michael Mauboussin, noted Wall Street strategist and thought leader, explains the power of the outside view:
…there’s another way to think about plans and predictions that doesn’t come naturally to us but is more robust. Psychologists call this the “outside view.” The outside view considers the problem as an instance of a larger reference class. Basically, the outside view imposes a fundamental question: “What happened when others were in this position before?” Research shows that the inside view often yields predictions that are too optimistic, revealing a form of overconfidence. The outside view generally tempers that overconfidence and provides a much stronger foundation for thinking about how the future might unfold.8
Experts are humans – easily influenced by behavioral flaws that degrade thinking. We need experts to focus on the outside rather than the inside view:
First, humans predictably and routinely deviate from many of the established rules of probability and logic. Humans have been shown to assign higher probabilities to specific events than the larger set of events for which they are also apart. Humans often neglect the base rate occurrence of an event, focusing on the idiosyncratic features of the particular case in front of them in determining its probability.9
Asking about base rates and historical context removes some of the systematic errors of experts. Most of these errors are compounded during uncertain environments, a regular occurrence in investing.
Instead of hoping that experts overcome these biases, we can circumvent them by properly framing our questions. We do that by asking about historical context and base rates rather than single point future predictions.
It only takes a subtle shift to shift the focus away from prediction to base rates. We should always start by asking how the current decision compares to similar past situations.
Tetlock recommends:
Tetlock recommends comparing a decision in a proper reference class. That is, how did similar situations play out. After establishing this base rate, we can look at the unique circumstances of the current situation in the right context. Otherwise we’ll tend to give too much importance with unique, but unimportant characteristics, while ignoring the powerful base rate.10
Another approach is to ask:
· What other decisions are similar to this one?
· What frames were used in those cases?
· What elements are different in this case?11
Here’s how we can frame investment-related questions to nudge experts to focus on base rates:
· Given the uncertainty with the COVID-19 pandemic, it’s tempting to try to predict the course of the virus. As I’ve argued, it’s much more useful to understand historical pandemics and develop a base understanding of how they have typically evolved. Only after understanding the base rate should we begin adjusting for unique aspects of the current pandemic.
· The same thought process takes place with manager selection. If a manager claims a superior process and ability, but has a poor long-term track record, emphasize the track record before believing the sales pitch.
· Don’t ask economists for their GDP growth forecast. Instead, ask what have GDP growth rates been historically given similar economic conditions.
· Don’t ask consultants for their specific asset class return forecasts. Instead, ask what historical returns were for similar environments.
It’s not necessary to completely disregard predictions and forecasts. We just need to ensure we are using predictions after building a proper base rate foundation. We ask better questions to experts when asking about base rates. Without proper questions, we’re hopeless to understand when experts lead us astray.
Remember this piece of advice from Tetlock:
Adopt the outside view and put the problem into a comparative perspective that downplays its uniqueness and treats it as a special case of a wider class of phenomena. Then adopt the inside view that plays up the uniqueness of the problem.12