If you thought we've been drowning in political opinion polls for a while now, just wait to see what the next three weeks brings as Election Day approaches. And the $64,000 question, as always, remains how accurate are these polls? This is a particularly perplexing query this year given the potentially biasing impact of the notorious Bradley Effect.
The Effect is named after Tom Bradley, the African-American mayor of Los Angeles who ran for governor of California in 1982. Bradley lost that election to a White Republican, George Deukmejian, despite the fact that Bradley had been leading by as much as double-digits in polls in the weeks leading up to the vote. Some, but not all, of the exit polls on Election Day also incorrectly foretold a Bradley victory. And thus, a political legend was born: polls often overstate support for a Black candidate because Whites don't want to admit their true preferences to pollsters.
As with any theory, assessment of the so-called effect requires efforts to falsify it. Closer scrutiny suggests that perhaps it is not as well-established as some would have us believe. Lance Tarrance, a campaign pollster for Bradley's opponent in 1982, wrote an interesting piece yesterday. In it, he explains that the polls for the '82 gubernatorial race were actually narrowing the entire week leading up to the vote, to the point where it was really too-close-too-call on Election Day. Furthermore, he suggests that any inaccurate conclusions based on exit polling simply reflected that Bradley did win the in-person vote, but Deukmejian pulled ahead once absentee ballots were counted.
Now I realize the dangers of basing general conclusions on the word of just one source (not to mention a source who was anything but impartial as the events in question transpired). But Tarrance's article is compelling and at the very least suggests that the evidence for the Bradley Effect is not as iron-clad as many in the media have made it out to be.
What other basis is there for talking about the effect? Other elections in which Black candidates received a smaller percentage of the vote than some polls would have predicted: Douglas Wilder in Virginia; David Dinkins in New York City. What's striking about this supporting evidence is that it doesn't seem to come from empirical analyses of how voters respond to poll questions. Rather, support for the Bradley Effect typically takes the form of corroboration through anecdote.
As my undergraduate research methods students are already tired of hearing me tell them, that's far from an ideal way to conduct hypothesis testing. OK, so there are some other instances in which polls exaggerated the support of Black candidates. But a rigorous assessment of the theory would also examine whether such disparities ever emerge for White candidates. And the answer, of course, is yes. For one, all you have to do is look back to the 2004 exit polling that pointed to a John Kerry presidency.
And what about Black candidates that did wind up with vote totals matching or even surpassing what the polls predicted? Sure, we heard a lot about the Democratic primaries in New Hampshire, in which Barack Obama lost despite polls predicting a victory. But Nate Silver's fivethirtyeight.com site reviewed the entire Democratic primary season, and found that if anything, Obama actually overperformed compared to polling predictions.
From a psychological perspective, I'm also less than convinced that we should expect much of a Bradley Effect this year, particularly with regard to pre-Election Day public opinion polling. Theories of modern racial bias suggest that people are loath to admit to the impact of race on judgment and behavior. But why would telling a pollster that you plan to vote for John McCain in a few weeks qualify on that count?
Concern about appearing biased is reduced or even eliminated when decision-makers have a race-neutral hook on which to hang their hats. And in this case, there are plenty of legitimate, unbiased reasons why a voter might tell a pollster that she plans to vote for McCain. Experience. Particular policy positions. The idea that anyone with 7 houses and 13 cars must be good at multi-tasking.
You might not agree with these reasons, but there are plenty of them out there for people to choose from. A White voter has little reason to think, I have to tell this guy I plan to vote for Obama or else he'll think I'm a bigot. Especially since the next questions in the survey will probably allow the respondent to explain her candidate preference. Yes, I realize that various polls have come out telling us that X% of Americans admit they won't vote for a Black presidential candidate no matter what. But these aren't the folks who are supposedly responsible for the Bradley Effect–this X% is the group who's completely willing and eager to admit to their bias over the phone to strangers.
To me, there seems little basis for concern that these pre-election polls are overstating Obama's support because of the so-called Bradley Effect. Certainly the effect is no more problematic than the other potential threats to poll accuracy, such as potential undersampling of first-time voters, African-Americans, and people without landlines–all factors that would seem to point towards underestimation of support for Obama.
Of course, I can't rule out the possibility that there will be thousands of White voters who go into the booth on Election Day planning to vote for Obama, who seize up at the moment of truth and pull the lever for McCain instead. But I remain skeptical. There just wasn't consistent evidence of this happening during Obama's primary run, and frankly it's mostly these same Democratic voters we're talking about when it comes to the Bradley Effect in November (since many Republicans have no plans to vote for a Democratic candidate regardless of race). Furthermore, to the extent that there has ever been empirical support for the effect, it seems to have dried up over the past decade or so.
Don't get me wrong, I think race influences much of how we see and think about this election, as it colors our social perception in life more generally. And I've written about race and the election here and here. But call me skeptical that there are substantial numbers of voters out there who won't vote for Obama because he's Black AND also tell pollsters they are voting for him because they're convinced that admitting their support for McCain will be a tell-tale sign of bias. And when it comes to Democrats who genuinely plan to vote for Obama, why would they suddenly have trouble pulling the trigger on that choice in November when they didn't in the spring primaries? Moreover, no one has ever accused the hard-core Hillary Clinton supporters of being shy about their Obama reservations.
Of course, I also figured that traditional pollsters' exclusion of young, cell-phone-toting voters meant John Kerry would outperform his numbers in 2004 (and I wasn't the only one). So I've been known to be wrong before...