How Social Science Tools Can Better Predict Election Results
Why we’re using ‘crowd wisdom’ and ‘truth serum’ to forecast the election.
Posted November 2, 2018
As a scientist, I would never claim to have a crystal ball for seeing the future. But when it comes to predicting elections, there are some non-traditional tools from the social sciences that can conjure a clearer forecast than traditional polls.
With less than a week before Election Day, my colleagues Wändi Bruine de Bruin, Henrik Olsson, Dražen Prelec, and I, are furiously analyzing polling data. In collaboration with the USC Center for Economic and Social Research’s Understanding America Study, we are investigating several novel methods to predict the United States House elections on November 6.
All of our prediction methods agree Democrats will win the popular vote, but the novel methods predict a smaller advantage for them than traditional questions do.
Traditionally, polls ask people who they are going to vote for. Predictions based on these “own- intention” questions can be biased if some people are systematically excluded from the polls, if people don’t want to admit who they’re going to vote for, or if they change their minds at the last moment.
Recently, we have tested another way of forecasting, by asking people who they think their friends will vote for. This knowledge, drawn from peoples’ understanding of their social circles, is uncontaminated by media reports and pundits’ opinions. Averaged across a national sample, these “social circle” answers gave good predictions of actual election results in the 2016 U.S. and 2017 French presidential elections .
Another method we are excited about is based on the so-called “Bayesian Truth Serum.”  Bayesian Truth Serum, or BTS, is a mathematical method that helps evaluate the truthfulness and accuracy of answers to questions that have no verifiable answer at the time of responding. Election intentions and predictions are the case in point. People’s answers about their own intentions and about their social circles can be more or less honest and accurate, but we cannot know by how much before the elections. BTS uses participants’ estimates about how other people would answer the same questions to calculate how much pollsters should rely on these participants’ answers.
We are also curious to know whether people can produce good estimates of how others will vote in their own state. Polls have asked people who they think will win , but not to predict what percentage of people would vote for different candidates.
So far, we have collected data from three 2018 survey waves, conducted between August 22-September 11, September 14-October 4, and October 15-October 29. In each wave, we interviewed more than 4,000 people across the country and asked them about their intentions and predictions for the 2018 U.S. House of Representatives elections.
Our results? People’s reports about how their friends will vote suggest a much smaller difference between Democratic and Republican candidates than the “own-intention” questions from the same poll. When we apply the Bayesian Truth Serum to own-intention and social-circle estimates, the margin of victory for Democrats narrows further. And predictions based on peoples’ estimates for election results in their state show an even narrower margin.
Our methods also allow us to say something about election outcomes in different states, although our study was not designed to give state-level estimates. For example, we have only around 30 participants each in Oklahoma, Louisiana, and Arkansas, and they happen to predominantly vote Democrat. However, they still report that most of their friends will be voting Republican, which is in line with predictions of much larger surveys conducted in these states .
To put our novel survey in context, the BTS predictions based on social circles come very close to current predictions from the site FiveThirtyEight.com, which aggregates predictions of hundreds of different polls . In other words, our methods might be uncovering the same information, but with less effort.
One positive thing for Democrats is the increase over time in reported percentage of participants’ social circle that are likely to vote for a Democratic candidate. Based on our experiences from 2016 elections , this could be an indicator of a mounting support for Democrats. Whether this will be enough for a “blue wave” or merely a “blue splash” remains to be seen.
 These national predictions are obtained by averaging predictions for each state, weighted by state population size. Survey results are from the USC Center for Economic and Social Research’s Understanding America Study, with at least 4,200 participants in each wave. 538.com predictions are based on their ‘classic’ predictions of vote shares in each district.
 Galesic, M., Bruine de Bruin, W., Dumas, M., Kapteyn, A., Darling, J. E., & Meijer, E. (2018). Asking about social circles improves election predictions. Nature Human Behaviour, 2, 187–193.
 Prelec, D. (2004). A Bayesian truth serum for subjective data. Science, 306, 462–466.
 Graefe, A. (2014). Accuracy of vote expectation surveys in forecasting elections. Public Opinion Quarterly, 78, 204–232.