Verified by 
This entry was inspired by this blog post by Sanjay Srinivasta. Sanjay has been at the forefront of improving psychological science, and his post was a faux syllabus for a course titled Everything is F****d: The Syllabus. Sanjay defines scientific practices as f*****d when they present "... hard conceptual challenges to which implementable, real-world solutions for working scientists are either not available or routinely ignored in practice."
Week after week, Sanjay's faux course presents readings arguing that some aspect of what we took for granted as "good" science in psychology is f*****d. This includes experiments, reviews, statistics, meta-analysis, replication, and more.
The thing is, I have been teaching two actual courses on basically the same topic since 2014. The one presented here is my undergrad course; another blog presents the graduate course.
My labeling is, perhaps, slightly less dramatic. Both courses are called:
The Psychology of Scientific Integrity
Scientific integrity refers to two concepts that advance the idea science should be about “getting it right”: 1. Personal honesty and trustworthiness in the conduct and interpretation of scientific research; and 2. Developing a body of conclusions that are valid and unimpaired. Statistics, methods, transparency, replication, and political bias have come to the fore as threats to the integrity of psychology and other scientific disciplines.
Why is it that studies that cannot be replicated become famous and influential?
Why is it that, even when a study can be replicated, the results in subsequent studies are often much weaker?
Why won’t some researchers let others analyze their data?
This course will provide some preliminary answers to these questions, primarily through readings and discussion. To be clear, though, this course is not anti-science. Its perspective is that the best antidote to bad science is not any of the vast variety of anti-scientific alternatives out there (post-modernism, social constructionism, supernaturalism, or anything else). The solution to bad science is good science.
The Final Project
The class gathers together about 20 papers from Journal of Personality and Social Psychology. Usually, this includes a total of about 60 separate studies. It then performs a series of what I call "forensic" analyses, most of which are available here:
The p-checker app can be used to answer questions like:
- Are the results too consistent to be credible?
- Is the research likely to be replicable?
- Is there really an effect there, or is the area riddled with publication biases and questionable research practices?
They also have the option of subjecting the studies to an "incredibility test." Results can be shown to be literally too good to be true via this test.
READINGS
However, the core of the course is the readings, the most recent incarnation of which are presented here. These are designed for advanced undergraduates. As such, they are not usually as technical as those you will find on Sanjay's page (which is a blog for researchers and graduate students). Some are mainstream news stories (NYTimes, Atlantic Monthly, etc.), but some are more technical. If you, my Psych Today Blog readers, want to know why some people think Everything is F*****d, I guess you are just going to have to bone up on some of the technicals.
Some last disclaimers for Psych Today readers:
- The syllabus is constantly in flux, and any given semester might not have all these readings and/or might have others not listed here.
- This really just barely begins to scratch the surface of such problems, which run far deeper than can possibly be addressed in a three month undergrad class. But it is a pretty good start, I hope.
Science is a Mess: Overview
- Benefits of Diversity. How arrogance has impeded astronomy.
- Nine Circles of Scientific Hell
- Lies, Damn Lies, and Medical Science
- Unicorns of Social Psychology
Wait! What is Science Supposed to Be: A Brief Overview and Refresher
Surgery? Don't Always Believe Everything Your Doctor Tells You
How Psychological Science Goes Wrong
- False positive psychology
- Questionable research practices
- A Brief Overview of How Psychology Goes Wrong (John Doris Guest Post)
- The Simpleminded and the Muddleheaded
- Reckoning with the Past
Its Not Just Psychology
The Twin Problems of Political Bias and Compelling Story-Telling: How Researcher Motives to Advance Ideological Agendas and Change the World Skews Their Research
- Political distortion of science: Stereotype inaccuracy?
- Liberal Privilege in Psychology
- Steele & Aronson (1995). Stereotype Threat
- Note: This is included as an example of how researchers claim to have found, and then promoted, a DRAMATIC WORLDCHANGING FINDING!!! ("remove threat, and black=white test scores"), when, in fact, they never actually found any such thing. See this post for the very gory details.
- Bem's advice on writing, as an example of how core training encouraged dysfunctional methods and statistics.
- Political distortions in the social psychology of intergroup relations
- Ideological bias in social psychological research
- Can high moral purposes undermine scientific integrity?
- Political diversity will improve psychological science.
- Interpretations and methods: Towards a more effectively self-correcting social psychology.
(Lack of) Transparency
And Yet, Maybe It is Not All Darkness and Despair
** At least Magician Will Fern admits he is just creating an illusion.
