Verified by 
Review of The 7 Deadly Sins of Psychology: A Manifesto for Reforming the Culture of Scientific Practice. By Chris Chambers. Princeton University Press. 274 pp. $29.95.
Scientists should be more like detectives than lawyers, John Johnson, a professor of psychology at Penn State University, has written. They are supposed to search for relevant data "that will help decide what is actually true, not just for data that supports a preconceived idea."
Unfortunately, according to Chris Chambers (a professor of cognitive neuroscience at Cardiff University), psychology, perhaps more than other disciplines in the behavioral sciences, "falls short of that standard." In The 7 Deadly Sins of Psychology, Chambers demonstrates that many psychologists, driven by the imperative to publish in prestigious journals, produce studies with weak data, faulty methodologies, and questionable conclusions. Passionate, provocative, and persuasive, Chambers' book is filled with information and insights about current practices in psychology — and offers recommendations to enhance transparency and reproducibility.
Chambers reveals that peer-review journals tend to publish studies on interesting subjects in which the findings are positive (i.e. A does, indeed, cause B) and are consistent with the researcher's hypothesis. Ironically, Chambers writes, the discipline that identified "confirmation bias" is "one of the most vulnerable to confirmation bias."
And so, "to ensure that they keep winning in the academic game," some psychologists engage in HARKing (hypothesizing after results are known). Chambers cites one study of 2,000 psychologists that found that 100 percent of them, at least once, excluded data after looking at the impact of doing so and collected additional data to reach a statistically significant result; 75 percent did not report all pertinent conditions in an experiment; more than 50 percent stopped collecting data as soon as they achieved the desired result.
And because very few journals will publish studies that seek to replicate the results of previous experiments, Chambers estimates that as many as 98 percent of findings "may be unconfirmed genuine discoveries or unchallenged fallacies."
Convinced that the credibility of studies in psychology are at risk, Chambers lays out a comprehensive and impressive reform agenda. He believes that journals should incentivize scholars to submit replication studies that confirm or contradict previous findings. Chambers recommends that investigators adopt Bayesian statistical methods to test hypotheses in more meaningful ways and master the difference between statistical significance and "real world" significance. He encourages collaboration among adversaries "to overcome ego driven biases." He wants journals to mandate data sharing. And he urges professional organizations to lobby journals to excise from the scientific record work which has unexplained anomalies in the data or failed the test of replication.
To make it more likely that empirical studies are deemed worthy of publication because of the importance of the issues they address and the rigor of their methodology, and not on whether the hypothesis has been confirmed, Chambers proposes that investigators submit a protocol to a journal before any experimental work is done. The protocol would be assessed for its scientific significance, clarity, methodological feasibility and precision, and for susceptibility to replication by other researchers. If editors approve, the manuscript would then be sent out for peer review, and, if given a favorable assessment, granted "in principle acceptance." Following data collection and submission, the study would be published if deemed consistent with the pre-registered proposal — no matter what the results. This approach, Chambers claims, is "a vaccine against bias."
Chambers' reform agenda is getting a hearing. He estimates that about 1,000 people attended his symposium at the Association for Psychological Science convention in 2014. Chambers can be tough on defenders of the status quo, but he seems sincere when he claims to be combatting the "dinosaurs within all of us — they are unconscious biases, fragile egos and propensities to cut corners." And the subjugation of "the true mission" of psychology "to the ideals of storytelling, glory hunting, game playing, and bean counting."
Led by a culture of transparency, psychology, Chambers concludes, can experience a renaissance. His "Registered Reports" may well be a good place to begin the difficult task of challenging the status quo.
