You don't have to lie to be a bully. If a kid says “You got picked last at recess.” “You aren’t popular.” “You can’t afford nice clothes.” etc., they’re still a bully, even if it’s true.
Replicators have recently been accused of bullying. Here’s how it works. Al publishes a study. Barb can’t replicate it. Barb publishes her failed replication. Al’s reputation takes a hit.
As Michael Frank points out, that’s not all that happens.
“The tone of the posts linked above is very professional, even if the overall message about the science is sometimes scathing. But one negative review typically spurs a host of snarky follow-ons on twitter, leaving a single research group or paper singled out for an error that may need to be corrected much more generally. Often critiques are merited. But they can leave the recipients of the critique feeling as though the entire world is ganging up against them.”
Is that bullying? Yes. Even if the criticisms are valid. A failed replciation is a painful thing and attacking it makes it worse.
As the Twitter reaction to this article about replication and bullying showed, replicators don’t agree with me. “We’re not bullies,” was the sentiment, basically, but with more indignation/derision.
But you don’t ask a bully whether they’re a bully. You ask the victim. If someone says you are bullying them, you should listen. What you should not do is exactly what replicators (and I consider myself one) have done: Rise up as a group and ridicule those who dare to say you’re bullies.
Let’s be clear: None of this is an argument against replication. Unlike a bully in middle school, replication has great value that, in the end, far outweighs the negatives. The replication movement is an overall good.
So what’s the point? Perception matters. If researchers feel bullied, the replication movement suffers. The real goal of replicators should be to change the culture of psychological science. If replicators are perceived as a gang of bullies, that’s probably not going to happen. It’s much more likely to succeed if efforts at replication are perceived as supportive and non-threatening.
That’s why I support Kahneman’s idea that there should be rules of replication etiquette. You don't convert people to your cause by attacking them.
Getting people who already believe in replication to do it is relatively easy. The hard thing is to convert people. If you really want replication to succeed, promote it by inviting people in. It’s not as fun as being snarky, and it’s more work. But it's time to step up and be supportive. Maybe the culture of psychological science will actually change.