Verified by 
How could the science at the core of the second most viewed TED talk in history turn out to be flawed? The talk in question is then Harvard Business School professor Amy Cuddy’s 2012 lecture on “Power Poses.” In it, she reports that adopting poses such as standing with legs spread and hands on hips changes hormone levels in ways that increase confidence and lower stress. This work has been cited many times, most notably in Sheryl Sandberg’s Lean In initiative.
In 2016, one of Cuddy’s co-authors, Dana Carney, admitted that “the evidence against the existence of power poses in undeniable,” and that she does not believe such effects are “real.” Of course, this admission does not prove that posture has no effect on a person’s physiology or psychology. Cuddy herself, who left her tenure-track position at Harvard, continues to maintain that the effects are real. More significant than the verdict on “power poses” is the underlying question: How can apparently reputable scientific findings lead us astray?
In the case of “power poses,” a number of factors appear to have been in play. One of the most important, Carney suggests, is commonly known as “p-hacking,” or cherry-picking data. This occurs when researchers select or analyze data in ways that make non-significant results appear significant. Another widely reported case of p-hacking involved a Cornell University food researcher, who resigned his post after six of his papers were retracted by the Journal of the American Medical Association.
To understand “p-hacking,” it is first necessary to understand p. P was born when statisticians began attempting to distinguish significant associations from background noise. The null hypothesis assumes that there is no significant relationship between two variables. To refute the null hypothesis and infer a significant relationship – say, between posture and hormone levels – researchers usually try to prove that it has a less than 5 percent probability of stemming from chance, or p-value of less than 0.05.
A problem arises when researchers begin interrogating large data sets looking for associations with a p-value less than 0.05. For example, if they tested for significant associations between 20 different variables, it is likely that at least one association would appear to be statistically significant merely by chance. Many studies purporting to show that factors such as diet, exercise, and alcohol consumption are good or bad for health are riddled by such problems.
But “p-hacking” is just the tip of the iceberg. Equally important is the tendency of scientific conferences and journals to report positive results. Simply put, a study that finds that posture influences hormone levels is much more likely to be published than one that fails to demonstrate such a relationship. This tendency is magnified by the fact that, when it comes to building the careers of researchers, positive results generally count far more than negative ones
Another factor is research funding. While funding for many types of research is scarce and competition is stiff, it is generally easier to obtain funding for research studies testing novel hypotheses than studies designed to confirm or disconfirm previously reported findings.
Add to this a similar bias in the popular press. When researchers and their institutions announce the results of studies, broadcast and print media are much more likely to disseminate positive findings. Should a study fail to refute the null hypothesis, or should subsequent investigators repeating a study fail to obtain similar results, press coverage is much less likely.
Underlying each of these biases toward positive results is an even deeper problem – namely, a fundamental misunderstanding of what science really is. Science is not a body of established and unquestionable facts. Science is more like a method of asking questions that – at its best – moves us closer to an understanding of what is really going on.
Understood in these terms, it is not surprising that in many cases, the received wisdom of science has turned out to be wrong – and not just concerning the result of particular studies, but entire scientific world views. The long-held idea that living microorganisms emerge from inanimate matter turned out to be wrong. The solar system turned out to revolve not around the earth, but around the sun. And the supposedly inviolable elements of the periodic table turned out to be mutually interconvertible.
Anyone with even a passing familiarity with the history of science would expect more such scientific revolutions in the future. For example, to balance the equations underlying current cosmological models, it is necessary to hypothesize the existence of “dark matter” and “dark energy,” which are characterized in part by their undetectability. Likewise, while “spontaneous generation” – the view that living microorganisms emerge from inanimate matter – was refuted two centuries ago, current accounts of the history of life assume that such a transition must have taken place at least once.
The point here is not that science is so hopelessly biased that no one should pay attention to it. After all, electricity really does power our devices, aircraft really do roam the skies, and antibiotics really can cure life-threatening infections. But as a way of asking questions, science is also subject to biases, and in many cases, such biases manifest as excessive enthusiasm for results that have not yet been subjected to sufficient scrutiny.
There is no panacea. To correct for such biases, we must never forget that that p-values are not truth, but potentially useful tools in its pursuit. Likewise, we must remember that researchers, funders, publishers, reporters, and members of the general public all operate with biases, including a bias toward positive results. Everyone wants to get in early on the next big thing – whether “power poses” or the elixir of life. As Plato said, only true understanding can protect us from wrong opinions.
