Is Statistical Evidence the Antidote to Anecdotes?
A recent meta-analysis suggests that it all depends on the issue.
Posted April 14, 2020
As we continue to cope with COVID-19 uncertainty, social media is abuzz with numerous recommendations seeking to influence people's attitudes about the situation and how they respond to it.
Should they be worried about whether this virus is going to kill them? Should they wear masks if they go out in public? Should they wash their hands 10 times a day? Should they let their neighbor come over to borrow some books? Should they create yet another My Sharona parody?
Most of these questions have no clear answer (except, perhaps for whether another parody of My Sharona is needed—the answer to that is no). There is an enormous amount of uncertainty because much of the information out there is conflicting – for example, it is unclear whether everyone should wear a mask, whether those who are potentially ill should wear a mask, and how effective different kinds of masks actually are. This conflicting information comes in all forms, from personal narratives to expert recommendations to fancy graphs and charts.
On Twitter specifically, I see tweets that include arguments that tend to rely on a single type of evidence, whether it be personal narratives, charts, or links to a single study. Comments on these tweets run the gamut of (dis)agreement with the tweeter's conclusions.
Undoubtedly, a lot of factors influence whether we'll be persuaded by different types of evidence presented on social media. A recent meta-analysis conducted by Freling et al. (2020), though, offers some potential insight. The meta-analysis examined several factors that influence our likelihood of being persuaded by anecdotal (e.g., personal stories, testimonials) or statistical (i.e., more systematic, empirical evidence) evidence.
In general, the results suggest that statistical evidence has a slight edge in general, but that doesn’t tell the whole story (and wouldn’t be all that worthy of a blog post). The most important results (which also happen to be relevant to the topic at hand) is that we're much more likely to be persuaded by statistical evidence when it concerns an issue:
- with little threat to us,
- that is non-health related, or
- that is less personally relevant.
In other situations (i.e., those opposite to the ones mentioned above), we tend to be slightly more accepting of anecdotal over statistical evidence. This is really important because it suggests we tend to be more evidence based when personal engagement is low rather than high. When personal engagement is high, we're less persuaded by empirical evidence and more persuaded by stories and anecdotes.
That high level of personal engagement fits with the current pandemic (go figure!)—it is personally-relevant, threatening, and health-related. Although Freling et al. didn’t test for possible interactions among these factors (limited as they were by prior studies), the current pandemic offers the trifecta of factors that make it less likely for us to be persuaded by statistical information and somewhat more persuaded by anecdotal evidence.
Yet, there's an enormous amount of COVID-19-related statistical information floating around social media (remember all those fancy charts and graphs I mentioned earlier). Regardless of the quality of that information (and it does vary), it is not likely to be as persuasive as more powerful anecdotal stories and narratives. Therefore, when it comes to COVID-19, if people have an argument to make and they want to make their case with statistics, they would probably be wise to supplement their statistical evidence with testimonials or personal narratives of some kind (i.e., personalize the message). Whether the goal is to persuade people about the (in)effectiveness of lockdowns or to encourage them to engage in particular behaviors (such as handwashing or mask wearing), if the argument is based strictly on statistics, it may not be as persuasive as an argument that includes at least some anecdotal evidence.