Are Economic Forecasts Overconfident?

Recently, David Leonhardt published a report suggesting that forecasts of economic growth are overly optimistic. His claim, if true, would signal a real problem with how our nation devises economic policy. If monetary and fiscal policies were based on perennially optimistic forecasts, then Congress and the Fed may be doing too little to support the economy.

But I’m not sure Leonhardt’s claim is true. I have been working with Sandy Campbell, a doctoral student here at UC Berkeley, to test for overconfidence in the Survey of Professional Forecasters, the same data David Leonhardt used for his February 8th report.

The forecasting data offer many ways to assess overconfidence. Forecasters make predictions quarterly, forecasting GDP growth and unemployment one, two, three, and four years ahead. The survey asks for both point predictions and full probability distributions. Knowing that this complexity afforded multiple approaches to data analysis and unsure that there was one obviously right way to assess optimism, Campbell and I pre-registered our analysis plans, including several different approaches to measuring optimism.

If these different approaches had yielded consistent results, we could have some confidence in the conclusion. However, inconsistencies in results between different approaches ought to undermine our faith in the truth and generalizability of the result.

What we find is complicated. Like Leonhardt, we find some evidence that forecasters’ point predictions, on average, optimistically overestimate annual growth in GDP by .28 percent. Their reported probability distributions imply a smaller overestimate. On the other hand, they pessimistically overestimate unemployment by .09 percent. The probability distributions imply an even more pessimistic error.

There was some reason to have expected that forecasters might be optimistic, given that it might feel better to expect a bright future. On the other hand, economics has a well-deserved reputation as the dismal science. More generally, the evidence for the claim that people are generally overly optimistic is not well supported by evidence. Sometimes, people tend toward optimism. In other circumstances, however, they employ defensive pessimism to motivate themselves to work hard or to avoid disappointment.

The more interesting result to emerge from our analyses is the consistent presence of another type of overconfidence: overprecision. Overprecision is the excessive certainty that you are right. A forecast can be both pessimistic and overprecise, such as when a forecaster predicts with confidence that the economy will shrink by 1 percent but it actually grows by 2 percent. We find strong evidence of overprecision in economic forecasting. Whether they are too optimistic or too pessimistic, forecasters are systematically too sure they know what is going to happen. On average, they report being 53 percent sure in the accuracy of their forecast, but they are right only 28 percent of the time.

Another potential interpretation of this result is to note that economic outcomes include some big surprises. For instance, forecasts for the second quarter of 2020, made a year earlier, did not anticipate Covid-19. As a result, those forecasts will appear, in retrospect, both insanely optimistic and recklessly overprecise. But if we admit that surprising things happen it then raises the question of why forecasts do not seem to reflect that reality.

Forecasters are not unique in their vulnerability to overprecision. Evidence suggests that most of us go through life too sure of the accuracy of our beliefs, perspectives, and opinions. This has profound consequences not only for planning economic policy, but also for our willingness to take advice and listen to those who disagree with us, as well as for innumerable private decisions. We could make better decisions if we were more open to listening to those with whom we disagree and considering the possibility that we might be wrong.