Why Do We Mispredict the Future?

By Hal Hershfield, Ph.D.
Consultant to Avantis Investors®

It’s the start of a new year, and I’m sure you’ve thought a decent amount about how the year will play out. (Don’t worry, this isn’t going to be yet another post about New Year’s resolutions!).

Consider, for a minute, something about your upcoming year that is marked by uncertainty. Maybe it’s the state of interest rates — will they go down? Or inflation — will it go up? Or maybe it’s something more personal, like the quality of your health or the quality of your relationships. Now, thinking about one of these domains, ask yourself what the chances are — from 0% to 100% — that things will work out a certain way.

As an example, I am uncertain whether my kids will start to get along more frequently. (I’m writing this over the winter holiday break, and they are spending a lot of time together ...bickering). What’s the likelihood, I wonder, that they will only fight one time per day instead of the usual much higher rate? One way to answer this question is to think about an optimistic best-case scenario, a pessimistic worst-case scenario and a most realistic scenario. And then, in theory, average them.

The best way to predict the outcome of a negotiation is to take the average of the initial offer from one side and the counteroffer from the other. In the same way, to improve the accuracy of predictions about uncertain events, you might take the average of your best-case and worst-case estimates.

If I think that there’s, at best, a 70% chance that my kids will start to get along better next year and, at worst, a 30% chance that they will, then perhaps the best estimate of their friendliness toward one another is something like 50%.

The Best-Case Heuristic

Yet, according to new research from Hallgeir Sjåstad and Jay Van Bavel, this isn’t the strategy that most people adopt.¹In their research, people were asked to make a prediction regarding how likely they thought it was that they’d be in a happy relationship in five years and how likely they thought it was that someone from their own political party would win the next presidential election.

However, the nature of the prediction varied. One group was asked to make an optimistic, best-case prediction. One group was asked to make a pessimistic, worst-case prediction. And a final group was asked to predict the “most realistic scenario.”

As you can see from Figure 1, whether it involves predictions about happy relationships or presidential elections, people fail to engage in an “averaging” strategy when trying to make realistic predictions about the future. Instead, their realistic predictions look an awful lot like their best-case scenario predictions.

Figure 1 | In Reality, Our Realistic Predictions Often Mirror Our Best-Case Predictions

Termed the “best-case heuristic,” people seem to hew close to their best-case prediction when attempting to make realistic predictions about possible future outcomes. In fact, the “most realistic” predictions are practically identical to the best-case predictions! The researchers found additional evidence for this tendency in studies.

Conducted in March 2020 for predictions about the likelihood of contracting COVID-19 by the end of 2020 and the waiting time for receiving the COVID-19 vaccine.

The mechanism at play may be something known as “attribute substitution,” where people replace the thinking that should go into making a realistic prediction with the thinking that lies behind making a best-case prediction. As Sjåstad and Van Bavel put it, “When people are asked to make their most realistic prediction, they might unknowingly be responding to what they want to happen instead because the best-case scenario comes first to mind.”

Implications for Financial Decision-Making

There are some interesting and important implications of this work for the world of financial decision-making. At the heart of many advisor-client interactions are conversations about hopes, fears, and expectations about the future.

When, for instance, might you want to retire? What sort of budget might you anticipate having in the next year? Will you want to stick with your current living situation for the next 10 years or change? You might notice that these questions, and others like them, rely on predictions.

To answer the question of when you might retire, you must predict, among other things, how much you’ll continue to enjoy working. You also must predict the likelihood that you’ll continue to be healthy enough to do the work that you need to do.

To answer the question of what your budget might be for next year, you must predict what your necessary expenses might be, as well as the level of inflation and so on. All these predictions can range from best-case to worst-case scenarios, but if, when making these predictions, you regularly gravitate toward best-case scenarios, you very well might find yourself in trouble.

In their work, Sjåstad and Van Bavel didn’t examine interventions that could be used to combat the likelihood of falling prey to the best-case heuristic. Could it help, though, if explicit questions were asked of decision-makers? Imagine you were asked, after making a prediction, whether it was based on reality or your hoped-for reality.

Might it change your predictions if you were directly asked whether your realistic prediction was the same as or different from your best-case prediction? Whatever the case, this work is a good reminder to ask yourself what you are basing your predictions on and whether they should be modified.