The Magic Wand of Psychological Research
The vital importance of random assignment to experimental conditions.
Posted Sep 11, 2019
I’ve been doing research in the behavioral sciences going back to my Research Methods class at the University of Connecticut in Spring 1990. If you’re counting, that’s like a zillion years ago. I have come to absolutely love behavioral science research and literally could do this stuff all day long.
I’m currently teaching an advanced research class related to the topic of positive evolutionary psychology (which relates to my new book on the topic, co-authored with Nicole Wedberg). The students and I are developing research projects to test questions related to positive evolutionary psychology, which seeks to use the power of evolutionary psychology to inform the positive aspects of the human experience.
So far, it is a ton of fun. Earlier this week, the issue of random assignment to experimental conditions came up. I referred to this process as the “magic wand of research psychology.” The powerful nature of random assignment to experimental conditions is one of those ideas that many people often don’t fully get or appreciate, even after receiving a solid education in psychology.
This post is designed to simply and clearly explain what random assignment is all about.
The Problem of Confounding Variables
Suppose that you are an educational psychologist working for a school district somewhere in the U.S. and you are part of a team asked to design and implement a study to explore whether a new reading program pushed by some academic publisher leads to increases in reading comprehension among 6th graders in your district.
Your general plan is to compare this new reading program with the existing program—which is, by the reports of many teachers, simply not that great. Good news: The district just told you that you have been assigned a paid intern, Ashley, who is a bright undergraduate psychology student at a local university. She is being paid to help you with this project.
So far, so good.
You sit down with Ashley and explain the point of the research and the task at hand. You ask her to develop a detailed proposal and bring it back to your office tomorrow. Dutifully, she complies.
Her research design has two groups. The first includes 50 sixth-graders who will receive the standard (already-in-place) reading program. The second group includes 50 sixth-graders who will receive the new “experimental” program.
You look things over and then ask Ashley this: “How are you going to determine which students are in which conditions?”
Ashley clearly hadn’t quite thought about that before. Hmm. She thinks for a minute. Then she suggests this: How about if we put all of the girls in the study into the experimental group and all of the boys into the standard group?
Stop right there.
There’s a problem. And the problem is this: Suppose that you go with Ashley’s idea and run that study. And then you come to find that the participants in the experimental group, who get the new reading program, score higher than do participants in the other group in terms of reading comprehension.
Given your research design, I’m going to venture that there are two competing explanations for such a finding:
- A. Possibly, the new reading program is just the best. After all, in your study, participants who used it did significantly better in reading comprehension than did the participants who used the old, standard program. But there is another possible explanation. . .
- B. According to the results of years of research (see Reilly, Neuman, & Andrews, 2018), girls generally (on average) outperform boys on measures of verbal ability, including reading comprehension. Thus, given the methodological design, it is perfectly possible that the only reason that participants in the experimental condition outperformed participants in the standard condition is that the experimental condition included participants who were all girls. Perhaps the relatively high performance of participants in the experimental condition had nothing to do with the details of the new reading program at all.
In this case, we would say that the biological sex of the participant, whether they are boys or girls, is a confounding variable. Such a variable co-varies with one of the variables that you are trying to manipulate and isolate in a way that you are not able to disentangle the differential effects of this confounding variable compared with your experimental variable.
Put simply: Was the high performance in reading comprehension due to quality of the new reading program or was it due to the fact that all of the girls in the study were in the "new reading program" condition? You just don't know.
The Magic of Random Assignment
In fact, you could probably think of a whole bunch of variables that may also relate to strong reading comprehension, separate from the details of the new experimental program. A student’s GPA is likely predictive of reading comprehension. So is his or her score on some reading aptitude test that was taken the prior year. The educational background of a kid’s parents may indirectly relate to reading comprehension. Time spent in libraries might related to reading comprehension. And so forth. In fact, there are probably dozens of confounding variables that might make it impossible for you to isolate the effects of experimental variable, which would be necessary to see if the new reading program, separate from anything else, leads to improved reading comprehension compared with the standard program.
Fortunately, there is a very simple and powerful solution: Random assignment to experimental conditions. Imagine that you assign the participants to the conditions not by some conscious process or choices, but, rather, by a coin flip or some other random process. Such random assignment will, on average, make sure that the kinds of people in one condition are similar, on average, to the kinds of people in the other.
Random assignment to conditions will (especially as the number of participants in the study increases) make sure of things like the following:
- The number of boys and girls will be roughly equal across the groups.
- The socioeconomic backgrounds of the two groups should be similar on average.
- The average GPAs of the students across the two groups should be similar.
- The total number of books read in the last year should be similar across the groups.
… and so forth.
Think about it: Random assignment to experimental conditions is incredibly powerful. This one simple-seeming process has the capacity to resolve the issue of confounding variables, allowing you, the researcher, to see if your actual experimental variable (which is all you really care about from the perspective of your research project) is having significant effects that are consistent with your hypotheses.
Yes, random assignment is the magic wand of scientific research methods.
In a strong psychology program, students learn all kinds of things about the scientific process that underlies behavioral science research. While there are many important ideas and processes that emerge in such an education, few rival random assignment to experimental conditions when it comes to simplicity and influence when it comes to designing good research.
If you’ve done a solid job of randomly assigning participants to experimental conditions in your study and someone asks a question about some confounding variable that was addressed by the random assignment process, you can just smile and say, “Oh, that’s not a problem: We randomly assigned participants to conditions.”
David Reilly, David L. Neumann, Glenda Andrews. Gender differences in reading and writing achievement: Evidence from the National Assessment of Educational Progress (NAEP).. American Psychologist, 2018; DOI: 10.1037/amp0000356