Why the Polls Were Wrong: Social Desirability Bias
Why did people deny who they were voting for?
Posted November 23, 2020 | Reviewed by Lybi Ma
Due to the COVID-19 pandemic, I have recently been spending time in a rural area of Pennsylvania, surrounded by lots of cornfields, little COVID-19, and more Trump than Biden signs. A few days before the election, I asked two acquaintances here whom they were going to vote for. They knew I supported Biden. “I’m undecided,” they each independently said, looking around a bit uneasily. They both ended up choosing Trump. I was struck, though, that neither initially wanted to admit what he really thought.
As election night revealed, they were hardly alone. Almost all major pollsters were wrong, predicting far more support for Biden than he, in fact, received in Florida, Pennsylvania, and other key states. Why were the pollsters so off?
My friends suggested the reason — called “social desirability bias” (SDB). This bias occurs when survey respondents give answers that differ from their real attitudes because they want to look better to others or feel good about themselves. Respondents thus underreport racist attitudes, substance use, smoking bankruptcies, and abortions.
When I asked one of my friends here why he voted for Trump, he shifted uncomfortably in his chair and stammered out, “Well, I don’t agree with everything he says, but my 401K is up!” “Aren’t you concerned about all his corruption and lies?” I asked. “All politicians lie. Biden is corrupt, too.” “I’m wondering where you get your news. Do you listen just to Fox?” “Oh, no,” he said, though I was unconvinced. “I listen to both sides. But,” he quickly added, “both sides are biased.”
When I asked another friend here why he supported the president, he replied, “Trump’s a jerk, but he's standing up to China. And why should my taxes pay to protect other countries? Let them protect themselves.” He, too, moved about in his seat awkwardly. “But I don’t like talking about politics,” he said, ending the conversation.
Some pundits have mentioned “Shy Trump” supporters, but these two men and many others are hardly shy; they carefully calculate the response they think they will encounter.
Psychologists have studied this phenomenon. For instance, one study conducted at the peak of the HIV epidemic asked gay men whether they always carried a condom as prevention against the virus. Many said they did. The researchers then asked each such respondent to show them the condom now. Only a handful of interviewees could remove a condom from their pocket, wallet, or backpack. Most did not, in fact, have one.
These interviewees and the two men here in Pennsylvania all gave what they felt were socially acceptable, responsible answers, even if these weren’t true.
Psychologists have developed scales for detecting such bias, asking respondents how much they agree or disagree with statements like, “When I have made a promise, I keep it — no ifs ands or buts.” “In conversations, I always listen attentively and let others finish their sentences.” And “During arguments, I always stay objective and matter-of-fact.”
No one, I suspect, would honestly fully endorse all these statements. Anyone who completely agrees with these comments is revealing SDB. Yet over 99 percent of psychological studies fail to use such a scale; those that do find that this bias affects the results almost half the time.
In this election, few pollsters tried to detect this tendency. In 2016, after pollsters grossly underestimated the support against Hillary Clinton, the American Association for Public Opinion Research issued a report on the failure and considered but rejected SDB as a factor. The organization argued that respondents would reveal their true attitudes in online surveys and since these also showed Clinton ahead, SDB was not involved. But SDB arises in online surveys, too.
This year, a handful of pollsters sought to discern this phenomenon. Arie Kapteyn, a University of Southern California economist who oversaw USC-Los Angeles Times polls, found that interviewees said they favored Biden over Trump by 10 points. But when he posed a “social-circle” question — “Who do you think your friends and neighbors will vote for?” — Biden’s lead dropped to 5 or 6 points. This question thus helped reveal SDB, but clearly also under-gauged Trump’s support.
Why did this social circle question work only partly? Because, as my two acquaintances here suggested, these voters are not only concerned with how they will appear to others, they are also, at some level, uncomfortable with their own choice.
Many Biden supporters see Trumpsters as evil, but these two men are decent people with whom I’ve had drinks. They are not evil. Still, in the end, they decided that saving money for themselves outweighed the president's egregious behavior. Ultimately, greed, not generosity, fueled their decision. Nonetheless, they were not proud of these emotions. Who wants to admit: I’m voting based on greed? They initially avoided my question about their vote, not wanting to think or talk about it. It was easiest to say they were undecided.
These internal conflicts are not easy to gauge, but pollsters, psychologists, and other researchers need to work further to do so.
In the future, we should also try to understand, engage with, and address these attitudes, this underlying discomfort, and urge such people to vote to help our country as a whole, not be selfish. Campaign messages could exhort: let’s work together to benefit each other, not just ourselves.
After the election, one friend came by. “Congratulations,” he mumbled, his head hung low. I didn’t ask him why he had initially avoided my question. I’m not sure he would have answered, or done so truthfully if I had.
(Note: This essay also appears in TheHill.com)