Just as all the children in Lake Wobegon are above average, a large number of psychotherapy techniques nowadays are characterized by their fans as "evidence-based." This sounds good, and most people have heard it enough to recognize that "evidence-based" must be something better than
"not evidence-based". Evidence is something you have to have for success in court, and none of us would want to be such fools that we believed everything without any evidence at all. But exactly what does evidence have to do with psychotherapy?
During the last 15 years or so, a movement in the medical world began to emphasize the need for appropriate research to show that treatments were effective-- that they actually accomplished what they were intended to do. The thrust of this movement for evidence-based medicine was that treatment for disease or injury should focus on the methods with the best indications of past success, not on anyone's personal preference, tradition, or old habits. From the medical world, the idea of using methods supported by good research evidence came into psychology as well. Evidence-based practice has become a goal in many fields.
But there are very few treatments such that we could say, "this one has perfect evidence supporting it; that one has no evidence at all. The first one is evidence-based, the second one is not." This is because there are many kinds of evidence that we could bring into a discussion about the effectiveness of a treatment. Even the much-maligned testimonial is evidence of a sort, although not a very convincing sort. In fact, we need to think not just about the amount of evidence for a treatment, but for the kind of evidence: what is called the "level" of evidence, as we rank different kinds of evidence from the very powerful to the very weak.
Professionals working in medicine, psychology, public health, and a number of other fields have generated a number of ideas about the factors that make for a high or a low level of evidence. Generally, authors agree that research evidence involving randomized controlled trials is the most compelling of all, because randomization ( the assignment of people to treatments on the basis of numbers alone, not because of their individual characteristics) does all we can do to exclude the confusion of confounding variables, that might make us think a treatment was effective when it was not. Under the many circumstances that make randomization impossible, the best approach might be clinical controlled trials, where individual wishes or circumstances place people in one or another of the treatment groups--- but these arrangements can make it difficult to know whether a treatment that worked for the kind of person who chooses it would also work for other people.
There are presently at least half a dozen ways to classify the level of evidence of research on treatment outcomes. With my colleague Monica Pignotti, I discussed some of these in two journal articles (Pignotti & Mercer [2007]. Holding Therapy and Dyadic Developmental Psychotherapy are not supported and acceptable social work interventions: A systematic research synthesis revisited. Research on Social Work Practice, 17, pp. 513-519; Mercer & Pignotti [2007]. Shortcuts cause errors in systematic research syntheses: Rethinking evaluation of mental health interventions. Scientific Review of Mental Health Practice, 5(2), pp.59-77). We suggested that treatments supported by randomized controlled trials should be designated "evidence-based", and those supported by clinical controlled trials termed "evidence-supported". Both of these are high levels of evidence, and with respect to psychotherapy, we need to remember that randomization can be very difficult to do. We also suggested a classification for treatments that were supported only by case studies or by other studies with various weaknesses: these would be called "evidence-informed". We created a fourth category, "belief-based", for methods that were based on a theory or assumption but which had no usable research evidence to support them. Finally, we added the category "Potentially harmful interventions" for those that had shown evidence of injury to persons receiving treatment, or that had manuals or other materials suggesting the possibility of harm (for example, materials that advise restraining children in the prone position).
These basic characteristics of different levels of evidence are not the only things required before a treatment can be called "evidence-based" or categorized in another way. For the higher levels, the research supporting the treatment would have to be replicated independently-- that is, by researchers who did not have the personal interest that the initial investigators usually have. There would need to be a manual for the intervention, or some other way of assuring intervention fidelity (that the treatment is carried out in the same way each time). Evaluation would need to be done "blind", with testers who are ignorant of the treatment an individual is receiving, so their assessments are not affected by their expectations. Statistical analyses would have to follow the rules agreed upon by researchers. In other words, the research and the reports of its results would all have to be done correctly before they were considered to be helpful in deciding whether a treatment was "evidence-based" or should be categorized differently.
Next time: Why "before-and-after" studies aren't good enough.
