Weather Forecasting in a Troubled Climate
Artificial Intelligence isn't ready to replace skilled meteorologists
Posted Apr 04, 2018
This essay is a collaboration with Robert Hoffman, and is largely based on Hoffman et al. (2017) Minding the Weather: How expert forecasters think. An earlier essay touched on some of the issues regarding the way skilled chess players and weather forecasters have learned to use Artificial Intelligence. This current essay goes into greater detail the ways that Artificial Intelligence is making greater contributions to meteorology.
Technophiliac pundits love to proclaim these days that the computer will eventually replace the human weather forecaster. This unbridled enthusiasm is justified primarily by noting the impressive advances in memory and computing capacity. The pundits’ misunderstanding of computer technology is apparent: All one needs is more number crunching, and Lo! A miracle will be performed.
For one thing, if it weren’t for human expertise, the computer models would not exist. So the fact that the computer models help forecasters at all represents a huge accomplishment for humans. For another thing, computer models do not produce weather forecasts. They generate predictions of the values of certain atmospheric parameters, such as surface temperatures, wind directions at various heights in the atmosphere, etc. It takes human expertise to go from the model outputs (along with other massive data that are available), and produce a meaningful forecast that helps people make sense of the weather and take appropriate action (see Kerr, 2012).
Forecasts have been improving in their value and accuracy for decades, and one part of the reason is that the computer models have been getting better. But when one looks under the hood, one sees that the things that the computer models are good at are also the things that human forecasters are good at.
The things that computer models are not very good at are less noticed in reports of the comparison of human and computer predictions. One exception is hurricane track forecasts, which are sometimes called “spaghetti graphs.” The different models sometimes generate different track predictions. But for many hurricanes, the models do converge, and hurricane analysis by computer models has shown great improvement in recent years.
But this is not a situation in which it is getting harder and harder for the human is to “improve” on the computer outputs, or produce forecasts that “beat” the computers. As I discussed in a previous essay, The Age of Centaurs, it is not productive to have a competition between the human and the machine. Forecasters use the computer models for what they are, tools in a very large toolkit. A saying in meteorology is “You cannot make a good forecast using the models unless you can make a good forecast without using the models.” Forecasters improve on the computer outputs by somewhere in the range of 10-25 percent (sometimes more), depending on which parameter is being compared.
The computers also depend on the humans. Human expertise is needed to adjust the inputs to the computer models to compensate for various tendencies the models have to over-or-under predict certain parameters under certain conditions. Humans also have to evaluate the outputs of multiple models (there are a good many of them) and determine which of them is the “preferred model of the day.”
We should not value number-crunching more highly than the human’s capacity for reasoning. Weather forecasting would not be possible without human-machine interdependence. We need more human expert forecasters, not fewer. What we now know about expertise can be leveraged into the training of forecasters, without doubt.
As Pearson calculated in 1978, the average per capita cost for the National Weather Service comes to about what you would pay for a large hamburger, fries, and soft drink. Correcting for the increase in the U.S. population (to about 320M today), the current NWS budget of about $972M translates to a per capita cost that is about that of a hamburger alone. Our current political “climate” is one in which economic and political agendas promote misinformation regarding climate change. Not only do we need more expert forecasters (e.g., Hoffman et al., 2014), we need expert forecasters to wield a stronger voice in public discourse (e.g., Collins & Evans, 2017).
Collins, H. & Evans, R. (2017). Why democracies need science. New York: John Wiley.
Hoffman, R. R., LaDue, D., and Mogil, H.M., Roebber, P., and Trafton, J.G. (2017). Minding the Weather: How Expert Forecasters Think. Cambridge, MA: MIT Press.
Hoffman, R.R., Ward, P., DiBello, L., Feltovich, P.J., Fiore, S.M., and Andrews, D. (2014). Accelerated Expertise: Training for High Proficiency in a Complex World. Boca Raton, FL: Taylor and Francis/CRC Press.
Kerr, R.A. (2012). Weather forecasts slowly clearing up. Science, 38, 734-737.
Pearson, A. D. (1978). Meteorological Big Mac. Editorial, The Kansas City Star. Reprinted in L. Snellman (Ed.), Forum, National Weather Digest, 3, pp. 2-6.