The Skeptical Sleuth

Applying a healthy dose of skepticism to new findings about health and psychology.

Try non-inert placebos

One of the things with placebos is that they're often called "sugar pills" But people in studies can often tell whether they're getting the placebo or the real medication, because the real medication causes actual side effects (as opposed to sugar pills, which don't).

I'd like to see every study use placebos that cause some kind of side effect. Load 'em up with niacin for a niacin flush. Or something that causes dry mouth. Then let's compare these placebos to the medication being studied. Methinks we're going to find even more placebo effects.

Kirsch's reasoning

My understanding of Kirsch's claim is not the effect of taking antidepressants is not clinically meaningless, just that it is not clinically more meaningful than placebo pills. The conclusion he draws is that taking antidepressants makes people feel better because of the placebo effect, not due to the active chemicals in the drug. This still means that taking antidepressants makes people feel better. He never claims that just taking sugar pills will have similar effect. What he does claim is that when people are given placebo sugar pills, told that they are just sugar pills, but research suggests (and it does) that even taking inert pills are helpful for people with depression, they show large effects and don't have side effects.

In regards to psychotherapy for treating depression, Kirsch does equate psychotherapy to a placebo, with their connecting theme being that both psychotherapy and plaecbos make people feel more hopeful that they can and will get better, and therefore help them do so. He identifies hopelessness as the key component in maintaining a depressive episode, and psychotherapy as a "placebo" to help people me more hopeful. In his book I believe he is pretty up front about that, and when practicing, explains psychotherapy in that way, basically that psychotherapy's primary mechanism of change is placebo (for depression at least). He believes that doing psychotherapy is more ethical than prescribing antidepressants because psychotherapy is a placebo treatment by his definition and upfront about it ("in psychotherapy we are going to work together to support you becoming more hopeful" , while antidepressant treatments work by having to mislead people.

@Cynthia - There are studies that do compare antidepressants to what are called "active placebos" as opposed to "inert placebos." Kirsch's analysis of them showed that there is no difference between placebos and antidepressants when antidepressants are compared to active placebos. This supports his claim that it is the side effects of medications that make people who take them believe they are working, so they start to feel better because they think they are taking something chemically helpful. It also explains why perhaps drug effects in antidepressant vs inert placebo studies show some antidepressant effect, because participants and doctors can and do break the blind by identifying side effects, so people know they are on antidepressants, not just hopeful that they are, so they are more confident they will feel better.

You've missed a major point: Risk-benefit

Kirsch's analysis has a crucial corollary in that when the benefits of antidepressants are overstated beyond placebo, a proper risk-benefit assessment cannot be conducted to decide whether a patient should be subjected to medication.

Psychiatric medications incur substantial health risks, increasing the incidence of diabetes, stroke, heart problems, and osteoporosis -- among other serious medical problems.

While not entirely risk-free, psychotherapy does not incur the same health risks.

It's by obscuring the risk-benefit balance and making doctors into patsies that pharmaceutical companies have been able to make billions of dollars on unnecessary prescription of medication.

Kirsch's work uncovers the smoke and mirrors used to push the commercial agenda.

Would that psychotherapy was also overprescribed! As a placebo, it might have done some good and much less bad than the excessive distribution of risky psychiatric drugs.

Kirsch has weighed in on this before, see below

Thanks for this interesting and provocative analysis. You raise an excellent point – what would Kirsch say if asked to defend the reality that the psychotherapy-placebo difference is similar to the antidepressant-placebo difference? Fortunately, we don’t have to speculate because he directly addressed this issue in a 2002 paper (Kirsch, Scoboria, & Moore, “Antidepressants and Placebos : Secrets, Revelations, and Unanswered Questions,” in Prevention and Treatment). I quote the key paragraph below:

“It may well be that the effects of psychotherapy are due to expectancy, conditioning, and other psychological factors that have been hypothesized to be the basis of placebo effect (Kirsch, 1997). Indeed, changing maladaptive expectations is an essential cornerstone of cognitive therapy. But this does not negate its effectiveness. The contention (e.g., Salamone, 2002; Thase, 2002) that the logic of placebo-controlled evaluation should be extended to psychotherapy is mistaken (Kirsch, 1978). In drug research, placebos are used to distinguish the pharmacologically produced effects of substance administration from its psychologically produced effects. However, there are no pharmacologically produced effects of psychotherapy. The effects of a psychological intervention can only be due to psychological factors. Control conditions consisting of alternative psychological treatments (including those erroneously called placebos) are useful in establishing which psychological factors are important in producing the effects of a particular therapy but not in evaluating the efficacy of that treatment. Like placebos, effective psychotherapies may accomplish their effects by changing expectations (Kirsch, 1985, 1990, 1999), but unlike placebos, they do so without deception.”
To summarize, the observation that a medication fares little or no better than a placebo pill is devastating; it indicates that the chemical in the pill is therapeutically inert. Alternatively, comparing psychotherapy with a placebo pill is both illogical and less threatening because psychotherapy is supposed to work via psychological factors (like the placebo effect).

Your analysis also ignores, as Altostrata pointed out, the critical issue of risk-benefit analysis. Compared to antidepressants, our best psychotherapies are of approximately equal short-term effectiveness. But why stop at this one metric when so many others are at least as important? Relative to antidepressants, psychotherapy is substantially more effective in the long-term after treatment discontinuation, produces no adverse effects vs. the many and often severe adverse effects of antidepressants (e.g., sexual dysfunction, suicide, blunted emotional experience), is more cost-effective in the long-term, and does not induce a state of biologically-induced chronic depression that renders users nonresponsive to subsequent medication treatment in 30%-50% of cases (El-Mallakh, Gao, & Roberts, 2011).

Kirsch’s basic point is that given the uninspiring efficacy of antidepressants relative to placebo, and the existence of alternative treatments (e.g., physical exercise, cognitive therapy, behavioral activation) that work just as well in the short-term, work better in the long-term, and have no adverse effects, why should antidepressants be considered a first-line treatment for depression? Among the statements of the many Kirsch-bashers and defenders of biological psychiatry, I have yet to see a compelling argument against this point. In fact, I have yet to see this point directly addressed at all. You certainly won’t find it in the APA statement in response to 60 Minutes. The APA wants us to think that Kirsch is a liar and a fraud, and to forget about the fact that the findings he discussed have been largely replicated by numerous research groups using divergent methodologies and are generally well-accepted in the scientific community. The argument is not whether antidepressants are barely better than placebo, but what the significance of “barely” is.

Best Regards,

Brett

Long-term iatrogenic effects

Psychotherapy is unlikely to have a long-term negative impact (especially brief treatment) but taking a drug every single day is indeed likely to cause negative impact on the brain. When someone takes a drug every single day and it's an illicit drug, we expect brain damage. Why do we expect that an SSRI would not have this effect?

Psychotherapy: training how to be a depressive

Anonymous wrote:
Psychotherapy is unlikely to have a long-term negative impact (especially brief treatment)....

I'd definitely argue that assertion. Psychotherapy is an artificial relationship that can distort thinking, a one-sided contrivance disguised as intimacy. It can present the practitioner as nearly godlike while promising strength and autonomy. And most relevant here, it can foster self-obsession with wounds, defects, anger, sorrow and helplessness, actually habituating the client to be chronically depressed. disequilibrium1.wordpress.com/

Response to Comments

First, it is a pity that if you document Kirsch's shoddy science and wild inferences, you become a "Kirsch basher" and that for some people, that label filters any evidence that you present and any holes in his arguments that you uncover.

Many of these comments reflect a confusion between Kirsch's opinions about the data and the data themselves. His selection of the criterion of ES = 0.50 was arbitrary and one that many medications widely judged as effective for nonpsychiatric conditions would not meet. Similarly, many psychotherapies would not meet the criterion in comparisons with active treatment conditions, including pill placebo. If the performance of antidepressants is "uninspiring,” then so too is the performance of psychotherapy, unless someone wants to admit considerable inconsistency in application of a ES = 0.50 criterion.

But I don't see Kirsch or people buying into his interpretations typically wanting to extend his criterion. His assertion that the logic of placebo-controlled interventions should not be extended to psychotherapies is a rejection of the basis for the whole evidence supported treatment movement. Maybe he intends that, but it is a radical opinion that should separate him from many of the advocates for evidence supported treatment who uncritically endorse him just because he seems to take their side in the medication versus psychotherapy culture wars.

Think about the logic of claims that placebos with side effects are more effective than placebos without them. This has been contradicted in the literally thousands of studies in which drugs under development with appreciable side effects time and time again fail to be different in efficacy then pill placebos.

If one checks back to Kirsch's articles in Prevention & Treatment (1998, 2002), one can see an extraordinary consistency of his position, even when it was based on thoroughly inadequate meta-analyses that essentially avoided peer review. Similarly, one sees his past dismissal of critics and refusal to address their claims. To borrow a malapropism from Yogi Berra, “déjà vu all over again.” See my next blog for more extended discussion and documentation.

I hope my comments did not

I hope my comments did not come across as labeling you a "Kirsch Basher."I certainly did not intend to, greatly respect your work, and take seriously your concern about assessing psychotherapy by the same standard as pharmacotherapy for depression.

the ES=.50 is indeed arbitrary, but for assessing changes in depression severity on the HAM-D, it seems pretty reasonable. I agree that there is danger in preselecting effect sizes without a good understanding of the measures being used and the context of the the variables being assessed. Looking at the meaning of a .3 effect size (a common ES Kirsch found) in regards to the HAM-D, translated back to a raw score on the HAM-D, it is only 1.8 points beyond placebo on the 51 point measure. NICE's recommendation that at least 3 points (ES=.5) of change is required to show convincing evidence of clinical significance seems reasonable given the measure, that 3 points is only about 6% change, and differences less than that would be hard for patients/mental health professionals to clinically assess. With this in mind, a lessening of depression severity as indicated by a 1.8 point change on the HAM-D being attributed to a specific drug effect to me does not warrant the risks of being exposed to that drug, if psychological interventions can produce similar overall effects.

As a supporter of the EST movement, I think it is important to do controlled experiments on psychological intervention efficacy. If a psychological treatment only produces 1.8 point greater change over TAU, without increasing risks of adverse effects of deterioration, the extra reduction of severity of depression is valuable and preferable. However, if participating in both TAU and the other treatment being studied are both associated with big gains in depression reduction, the study may also provide evidence that both the studied treatment and TAU are effective for treating depression, while one may be more effective than the other (and considered more strongly as a firstline treatment). If an active treatment can only produce a 1.8 point greater change over no treatment, than the benefits of participating in the treatment over risks (cost, time, adverse effects, etc) may not be all that worth the trouble.

My biggest critique of the EST movement is the claims that ESTs have known mechanisms of change, when many ESTs (as listed in division 12 of APA's website), an only show that that participating in them show changes in outcomes. It could be that placebo is a big contributor to their effect, and it is tough to do placebo-controlled studies of psychological interventions. I believe that research on psychological treatments should be rigorous, experimental, and controlled as possible, but conclusions drawn from them should not extend beyond the methodological limits of the studies. I believe the debate about EMDR meeting criteria as an EST highlights this critique, as rigorous studies of EMDR have shown that participating in EMDR shows greater reduction in PTSD symptoms beyond no treatment and similar to comparitors, but many people are critical that the active/key component of EMDR, rapid-eye movement, is a mechanism for change. It may be a placebo effect. It nonetheless seems to work, and without increasing risks in comparison to other PTSD treatments.

I don't know the research that contradicts Kirsch's speculation that placebos with side effects (active placebos) have similar effects to antidepressant medication for depression, but he seemed to do a pretty thorough survey of the medication indicated for treating MDD. Perhaps for depression, and not for other conditions, this "side-effect effect" is present.

ES = .0.5 is arbitrary and many accepted medical treatments wouldn't make it

What is special about an effect size of .5 and why use it to mark the difference between effective/not effective?

Many accepted medical treatments do not achieve it.

Probably why NICE abandoned it as a standard.

As Kirsch well knows, mean differences in a clinical trial are not very predictive of individual treatment response. Can be greater or smaller.

I agree with you about the exclusion of EMDR from the label "evidence supported."

Psychotherapy on average has a modest effect (no effect by Kirsch's standard, and people can make intelligent, informed decision to opt for antidepressants as an alternative. In our German trial, evidence for efficacy of antidepressants with mildly depressed was better than evidence for behavior therapy. I would not necessarily recommend one over the other, but would respect patient preference and watch to see if chosen treatment actually worked. But for mild depression, best to wait before committing to intervention anyway.

ES as contextual

I agree that setting an ES=.5 as always being a medium effect size is quite arbitrary and definitely not correct. However, determining what amount of effect is worthwhile, worth the risk, or "enough" change, is always contextual, and is never static. Getting a statistically significant change with a p-value of <.05 does not necessarily say so much about to what effect a treatment is associated with a change in outcome. The drug trials show that small effects can be statistically significant without having much clinical relevance. I am not saying that the effects of drug effects in antidepressant trials is small because it does not have an ES of >.05, I am saying it is small because of on the particular measure of effect, the HAM-D, the size of effect, while statistically significant, just isn't clinically discernible (Kirsch comes up with 1.8 points).

I hold that the meaning of effect sizes are not standard, but in this particular instance of where an effect size of .5 corresponds to a change of 3 points on the HAM-D, I think that is a reasonable minimum amount of change to expect for drug effects to be in antidepressants, because anything less does not have a lot of value in practice nor worth the known risks. This is up for debate, but I think it is very reasonable.

As for how this relates to psychotherapy studies, if they use the HAM-D to study changes in depression severity as the primary outcome measure, then a similar effect size would be useful for determining clinical relevance, especially in comparisons against no treatment/the waiting lists. I mean, if engaging in a treatment cannot get depressed people to score 3 points less on the HAM-D than doing nothing, then even the study can show statistical differences, those differences aren't that meaningful and that treatment has not shown a lot of promise. I am willing to accept smaller effect size differences for studies comparing a newer treatment to an established EST, and for treatments that have shown no/little risks/costs. I strongly support evidence supported treatments and the movement behind them, but I think it is a little off-based to claim great effect of particular ESTs if the evidence that supports them does not show that effect. I think it is better to have evidence of small effect than no evidence to determine effect at all, but having the evidence does not make a treatment a necessarily good one (though without evidence no treatment should be implemented.) If psychotherapy studies are not showing great effect after collecting evidence, that is important to know.

To your point on pooling effects across studies. If doing meta-analyses of the studies used to seek FDA approval of drugs do not find a meaningful effect of drugs on average, then let's reanalyze the data to see how many people need to be treated in order for some to get meaningful benefit, and how many people experience side effects and deteriorate in the process, and determine if we think that is worth approving (and think about how we would go about determining that, which would come down to an arbitrary but reasoned cut off based on the context of the studies, the disorder being treated, the outcomes being assessed, the outcome measures use to assess those outcomes, and a benefit/risk ratio of the risks/costs that may be involved in a treatment.) I think antidepressant approval trials are perfect trials for analyzing effect across studies because they are designed so similarly and use the same outcome measure, the HAM-D.

Being evenhanded With the Hamilton Depression Rating Scale

We can jump to a discussion of efficacy as measured by changes in the Hamilton Depression Scale, but the conclusions will be the same. The measure has been established administratively by the FDA, but it has serious limitations as a measure of severity and of therapeutic change, as Darcy Santor and I demonstrated in an item response theory analysis of it and Michael Bagby demonstrated in a more systematic review. And the criterion of the three points difference makes no sense in a randomized controlled trial with mildly depressed primary-care patients like I did with Ulrich Hegerl and colleagues. But when we performed a meta-analysis of published studies of psychotherapy versus pill placebo specifically focusing specifically on the Hamilton depression scale, psychotherapy performed no better than antidepressants.

so if we limit our discussion To the Hamilton Depression Rating Scale, we again have to concede that antidepressants and psychotherapy performed similarly in comparison to pill placebo. I agree, differences are relatively small, but I don't see the basis for labeling them as Kirsch does for antidepressants, clinically insignificant or trivial. And to ply the label only to antidepressants is a matter of of an ideologue being inconsistent in the face of evidence.

We can convert effect sizes to the number needed to treat (NNT) a useful comparative statistic, that I routinely include in meta-analyses and systematic reviews. For a condition like depression, NNT = 7 for both antidepressants and psychotherapy versus pill placebo. One has to be careful about interpreting NNT out of context. Based on available evidence, my colleagues and I estimated that one would have to screen about 700 primary-care patients to get one of them to have clinically significant improvement, so the corresponding number needed to screen is 700 one clinical improvement, hardly a recommendation for screening.

For patients with hypertension being treated with a cholesterol-lowering drug in order to prevent cardiovascular event, the number needed to treat to avoid one event is 99.74 over 3.3 years, at least in one major trial.

Continuing

Thanks for engaging with me, Dr. Coyne.

I agree that the HAM-D is limited, but I do think it is important to not make claims beyond the evidence gathered. Finding richer ways to assess change in depression severity I think would be great and should be pursued and used, they just haven't been used by the FDA and the data we have on antidepressant outcomes for the approval process comes primarily from the scale

I still think we are talking around each other on the meaning of the effect of pill placebo. Pill placebo in the FDA approval studies (and other studies) for antidepressants is robust across scales of initial depression. Treatment with just pill placebo seems to be a pretty strong treatment. So if psychotherapy and antidepressant medications show similar effects to pill placebo, then the total effects of those treatments I think are strong. Kirsch only claims that because pill placebo duplicates at least 80% of effects of antidepressant medications, and the remainder of effect that could be attributed to the chemical compounds of the medication is clinically indiscernible (though statistically significant by conventional standards), the primary mechanism of change of antidepressant medications, which have robust effects, comes from placebo. The argument for being cautious about antidepressant prescribing, especially with milder depressions, is that the risks of adverse events are high, costs are often high. Additionally, the common narrative (though not necessarily the narrative of the academic community) is that the antidepressant medications work by drug effect, and the data just doesn't show that. However patients are told this by their doctors and other mental health professionals and believe it. The dilemma is that people who take medications experience strong benefit, but because that benefit comes from placebo and not the drug, the believed mechanism of change looks false. The fear is that the placebo effect will diminish if they know this. It becomes truth vs. benefit, and whether or not it is ok to maintain misinformation for patients benefit. This is not a clear cut debate, but I tend to side that the more truthful we can be with research, we will eventually pursue better treatments overall. Kirsch cited studies that people do respond very well to inert pills when told that while inert, people who take pills every day to treat depression do receive substantial benefits. Though unorthodox, the claim happens to be true (as seen in the data), avoids many of the risks of antidepressant medications, and is truthful. I think this is a more ethical way to alleviate depression symptoms than having to mislead people.

Kirsch does not separate placebo effect from psychotherapy for treating depression in the same way he does for antidepressant medications, the reasoning being that mechanism for both are considered psychological. I think there is a debate here in deciding whether this is fair to do or not, especially in light of the EST movement that tries to establish applied treatment efficacy and effectiveness beyond placebo or comparison treatments. I think we do agree that there are essential differences in researching efficacy or psychotherapies and drugs, but to what extent those differences mean that they can be judged by dissimilar standards is debatable. I believe Kirsch falls on the side that you don't have to give misleading information about psychotherapy outcomes or mechanisms for treatment to be effective and for the current ESTs there are very limited risks, so the placebo effect does not have to be separated from total effect (placebo and active mechanism) when determining treatment effect. Placebo is harder to differentiate in psychotherapies because of methodological limits, but I think this lends to Kirsch's argument that you don't have to disentangle them for psychotherapy effects for treating depression. So while drug effect of antidepressants are determined by separating that effect from the placebo effect, psychotherapy effects can be assessed by the additive effects of placebo and specific effects. I think this can be ethically applied in treatment, as I don't think anyone would say specific effects are all the effects of particular psychotherapies, and hope and expectancy of treatment effect plays a role all treatments. I would expect people to get less benefit from treatment if they do not believe that their condition can change or that the treatment they are in will be at all helpful than if they do. So no matter which EST a clinician utilizes in treatment, a clinician and truthfully and helpfully make a claim about the specific mechanisms believed to be involved in treatment and that because the data is strong for an effect, change/relief can be likely. I think this is an ok place to start from, but more research should be done to determine which treatments are most effective, cheapest, quickest, with fewest risks. The biggest problem I see with this is that the EST movement presumes that specific mechanisms of change are the primary components of change, and Kirsch's argument may make the specific effects of treatment less important if TAU or nonspecific treatments reproduce a substantial amount of treatment effect. I do think we should follow the data, and if that is true, it may be that placebo effects are the strongest or at least strong part of the effect of depression treatments.

I don't think it is necessarily problematic if psychotherapy treatments perform similarly to pill placebo for treating depression if the effect is still strong. It may make pill placebo the preferred first treatment for depression because it would be cheaper and perhaps quicker than psychotherapies, but because not all treatments work the same for everyone, evidence for certain psychotherapies that can produce similar effects allow for more treatment options.

I have found some of your logic in past posts and this one to come down to "If we judge antidepressants do not perform better than placebos, and psychotherapies do not perform better (or even as well) as antidepressants by the same criteria, then that would make psychotherapies not particularly effective. Because we believe psychotherapies to be effective, then we have to reject the interpretation of data that shows antidepressant medications do not meaningfully outperform antidepressants." Forgive my simplicity, but I hope I have laid out what at least I think why Kirsch thinks we may not have to assess them similarly. Even if we do, I think we need to risk the idea that pill placebo may be almost or as effective as many psychotherapies for various severities of depression. If our goal is enhancing health outcomes, easing burdens of and access to treatment, and keeping costs and side effects down, pill placebo may beat many psychotherapies if their primary outcomes are the same but some secondary outcomes for pill placebo are more valuable.

As for NNT and NNH (number need to harm), I agree that there are risks of using them as primary ways to assess treatment outcomes because like effect sizes their meaning and what people are willing to accept are contextual on the disorder, treatment, outcome measure, severity risks, etc. I raised the issue in response to your previous comment regarding that people's individual outcomes can vary a great deal from the averaged outcome of a trial group, I certainly don't disagree that people's individual experience often varies from the average effect across a group or even meta-analysis, but that we could use NNTs as an additional way to assess treatments, and that it is just as contextual/arbitrary, but can be well reasoned. Response/remission/limited change/deterioration percentages can also help look at the data in other ways. Perhaps looking at individual data might help better understand why some people respond better than others so that we can better target treatments, but to my understanding, individual response differences have not been able to be well predicted as of yet. (Kirsch may claim that those differences may be related to expectancy effects, placebo or even nocebo.)

Dr. Coyne, I want to thank you again for engaging. I really value your willingness to flesh out your arguments and to be critical of new claims in mental health. We are all served by discussion and debate of findings and the push for more refined research.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.
  • You may quote other posts using [quote] tags.

More information about formatting options

Jim Coyne, Ph.D., is a clinical health psychologist and Professor in the Department of Psychiatry at the University of Pennsylvania. more...

Subscribe to The Skeptical Sleuth

Current Issue

Dreams of Glory

Daydreaming: How the best ideas emerge from the ether.