If you follow politics at all, you have probably heard of the poll aggregators, most famously, FiveThirtyEight.com, run by Nate Silver. The brains behind the aggregators combine polling data from multiple sources to get more accurate superpolls.
These polls are much more accurate than single polls, and several of them are taken quite seriously, even by political scientists, as a snapshot of the race. As I write this, Obama has a 74.6% chance of winning the election, according to 538.
At the risk of sounding obvious, that’s not 0% and it’s not 100%. It’s somewhere between the extremes, which is exactly where it’s been all along. So should partisans (fans of one or the other candidate) be happy or unhappy with these numbers? And how happy should they be?
Despite the sophistication of the algorithms that go into the aggregator algorithms, they make no attempt to model what goes with probabilities inside the human brain. But perhaps they should.
Humans are very poor at understanding probabilities that are not 0 or 100%. Economic psychologists have used carefully calibrated methods over decades to measure exactly how poor we are, and it’s kind of shameful. But at least it’s systematic. We tend to underestimate probabilities greater than around 40%. We hear 74.6 and we think around 60. We are particularly bad at even higher probabilities. We hear 99.9% chance and we can’t wipe the sweat off our brows and relax. We hear, maybe, 80-85%. (Different measures give different numbers, so I’m guessing here, but the trends are all consistent).
Conversely, we tend to mentally overestimate probabilities lower than around 40%. Especially low ones. Like 1%. We hear that and we think 15%. This may explain why people are so susceptible to lotteries and careers in professional football – these things have very low success rates, but they are systematically over-estimated.
Things get even weirder. The frame direction matters too. So if hear that Obama has a 74.6% chance of winning, you are more pessimistic about his chances than you would be if you were told that he has a 25.4% chance of losing. Even though these are mathematically equivalent. Similarly, people understand statements like '3 in 4' differently from 75% and 3 to 1 odds.
In short, statisticians can build sophisticated models with stunning accuracy, but most of us can’t really understand the numbers they produce.