Verified by 
I’m not great at everything, but I do understand statistics pretty well. I took five graduate-level stats courses in a Ph.D. program at the University of New Hampshire and I have been teaching statistics, at both the undergraduate and graduate levels, since 1996. And I’ve written (along with Sara Hall) a textbook on the topic (Straightforward Statistics: Understanding the Tools of Research).
In teaching statistics for so long, I always find myself disappointed in the attitudes that many people hold about this area of applied mathematics. People often have themselves convinced that they “are not good at math” and that they “will never really understand stats.” This is a problem for two reasons: First, these attitudinal barriers get in the way of otherwise bright and confident people fully reaching their educational potential. Second, with so many people out there being stat-phobic, the job of researchers becomes almost too easy! If people out there are afraid of statistics, then researchers who have even just a few tools in their stats toolbox can really just say whatever they want—with the statistically phobic given little choice but to believe what they are presented with.
This post is designed to help address one of the major statistical concepts that often gets people to throw their hands up—statistical significance.
What is Statistical Significance? What does p < .05 mean?
Statistical significance, often represented by the term p < .05, has a very straightforward meaning. If a finding is said to be “statistically significant,” that simply means that the pattern of findings found in a study is likely to generalize to the broader population of interest. That is it.
For instance, suppose you did a study with 100 cats and 100 dogs. And you found that, in your sample, 80 of the dogs were able to be trained to go through a hoop and only one cat was able to be trained to go through a hoop. And suppose you ran some statistical test and found that p < .05. That would simply mean that the pattern you found, with dogs being better at jumping through hoops, is likely to be a pattern that holds across the entire population of dogs and the entire population of cats. Further, this statistical language implies that the probability of the pattern of findings from the study not generalizing to the broader populations of interest is very small—less than 5% (thus, p < .05)—with p meaning probability and .05 simply meaning 5%.
What is magical about 5%? Well, nothing really! It’s kind of a practical benchmark that statisticians have come to use as a standard over years and years and across lots of different disciplines. It’s a worthy question, but also a question for a different post as it raises a whole bunch of other, more complex issues.
Bottom Line
Statistics are tools used by psychologists and behavioral scientists. They are designed neither to be scary nor mysterious. They are straightforward mathematical tools designed to help us better understand the world. Statistical significance and its related term p < .05 are simple concepts—simply meaning that the pattern found in a sample likely generalizes to the broader population of interest that is being studied. There’s no abracadabra there!
References and Acknowledgment
Thanks to my graduate student, Vania Rolon, whose speech at SUNY New Paltz that was part of the ribbon-cutting for the newly renovated Wooster Hall, with a focus on her passion for teaching statistics, partly inspired this post.
