Anxiety Files

As a recent convert to CBT (I recently finished CBT training at the Beck Institute), thank you for such a thoughtful, eloquent, and respectful response to Dr Allen's blog about CBT. Whle I appreciate differences in opinion, Dr. Allen's blog was obviously erroneous but also disrespectful. His blog did very little to contribute to the on going dialogue about the merits of CBT and psychodynamic therapies. Thank you again for your response Dr. Leahy, as well as for your professionalism.

First of all, I am not a psychoanalyst by a long shot, so I'm not sure why you even think I am. The analysts' efficacy claims have been even more ridiculous than some of the CBT claims.

Second, I did not invent the term, "cognitive behavioral mafia," that's a term used by psychotherapy researchers in the Society for Psychotherapy Research, which I was a member of for many years. Leading trauma researcher Bessel van der Kolk couldn't get a psychotherapy research grant at one point because of it!

Are you saying new ideas should not be researched because we already have a perfect treatment modality in CBT? Please.

Thirdly, I did not condemn all of CBT, just pointed out its limits. I agree that it is quite effective for most of the conditions you describe - IF they are not complicated by severe personality pathology and a lot of other comorbid conditions. As a psychiatrist, most of the patients I see have many comorbid problems and are highly dysfunctional. Compared to the more severely disturbed patients, what you are describing are relatively straightforward problems.

Fourth, I am well aware of schema therapy, and it incorportates many ideas from other therapy schools, and it is not pure CBT by a long shot. (check out the work on schemas by Mardi Horowitz, who was trained as an analyst). I know Jeff Young, who had been a protege of Aaron Beck and was one of the cognitive therapists in the big NIMH collaborative study on depression, which incidentally also found interpersonal therapy equally effective to CBT in "depression." In fact, he told me many in cognitive therapy circles turned on him when he started to talk more about issues such as the effects of child abuse.

CBT is constantly evolving just like other forms of therapy and is incorporating many new ideas from humanistic and interpersonal therapies. I was commenting on the classic form in my blog. If you want I can furnish direct quotes from the books of Albert Ellis and Aaron Beck that back up what I said about what they have preached - one from as recently as 2007.

Fifth, Marsha Linehan also incorporates a lot of ideas in her treatment of BPD from other schools, whether she admits it or not. She claims DBT is "empirically validated" as a treatment for BPD, when in actuality it is empirically validated for only about two symptoms of the disorder. That is the kind of exaggeration of the research that I'm talking about. And she writes very little about what to do about one of her two postulated causes of BPD, the invalidating family of origin environment, so it's an incomplete treatment according to her own theory.

Sixth, outcome measures used in CBT studies are highly suspect. The Beck depression inventory, for example, does not discriminate between major depression and dysthymia. These syndromes respond very differently to both medication and psychotherapy.

Seventh, there are very few dismantling studies which tell us which interventions of the vast number used by CBT therapist are important and which ones are superfluous.

I could go on with a bunch more criticisms of CBT research, as I do in my book. I'm not saying that empirical research is not important, only that research results are being highly exaggerated by the "evidence-based" proponents. (The same thing is happening in drug research, by the way). And relationship variables, due to their enormous complexities, are impossible to measure with precision, so if they are measured at all in studies, they are highly general.

Thanks for the great article. For almost all of the issues my clients to come therapy to deal with, CBT is the approach most likely to give them a great of immediate relief, and to give them the greatest benefits over the course of therapy. Although that is my personal experience, the reason I use CBT is that's what the research suggests will be most effective, and that research is confirmed for me in practice every day. It is unfortunate that psychoanalysts fight so hard to justify a method that will take longer for the benefits to begin and continue considerably longer and at a much greater expensive to the client.

I am constantly amazed at the changes in people's anxiety levels and ability to cope with anxiety that can come out of only a single session of CBT. Although I have no experience with it, I can't imagine short-term dynamic psychotherapy can have such quick results, let alone psychoanalysis.

Perhaps psycholanalysis is effective for people with no serious mental health issues who are more interested in self-exploration and personal growth. Rather than being just for simple problems, when it comes to serious issues, it is a disservice to offer clients anything other than CBT or similar approaches that have strong empirical support.

Agreed wrt psychoanalysis and my comment, but when I wrote it I was under the impression you were an psychoanalist based on the opening sentence in which you were described as a psychoanalytic psychiatrist. Though using a straw man in somewhat appropriate to counter your CBT straw man. Your comments about CBT show a lack of understanding, and amount to little more than a statement along the lines that CBT, when it is conducted in a superficial manner by inexperienced therapists who lack a solid understanding of CBT, is not that valuable to people.

First of all I want to thank Dr. Leahy for his thoughtful, succinct and balanced response to Dr. Allen's thoughts. I find these type of discussions useful in that they provoke thoughtful responses on my part with regard to "best practice" and "empirically based treatment approaches." My introduction to CBT began in the early '80s. Since that time I have had the incredibly good fortune to meet many of the researchers, educators and thought leaders in psychology through several professional organizations including the American Nurses Association, the Association for Behavioral and Cognitive Therapy, the International Association for Cognitive Psychotherapy, the Academy of Cognitive Therapy and many more. Most recently I attended a large congress on Psychopharmacology coordinated by the Neuroscience Education Institute (NEI), under Stephen Stahl, MD. Dr. Stahl is an accepted world authority on Psychopharmacology. At the NEI Congress, CBT was repeatedly brought up as a solid treatment choice for many disorders either as first line, as an adjunct or as an alternative. Dr. Stahl's team of faculty used brain imaging scans to demonstrate the efficacy of cognitive and behavioral therapies on brain structures as well as on behavioral/cognitive outcomes. Very impressive indeed!

The more we learn about the brain, the more we need to focus on the neurobiological impact of our therapies. It is truly a blessing, as a clinician, to have the option available to use pharmacotherapy, psychotherapy or a combination of the two. I see rapid, exciting changes in patient symptoms and function.

The bottom line is that CBT is effective, well researched as Dr. Leahy as eloquently pointed out, when used by clinicians trained in the model. Perhaps Dr. Allen has had the misfortune of being exposed to one, or more, of the clinicians claiming to be a Cognitive Therapist because they use a few of the techniques as opposed to the full model?

Finally, I hope that opinions that are not science based, as in many of Dr. Allen's comments, do not prevent access to effective care by individuals that could definitely benefit from their use.

First of all I want to thank Dr. Leahy for his thoughtful, succinct and balanced response to Dr. Allen's thoughts. I find these type of discussions useful in that they provoke thoughtful responses on my part with regard to "best practice" and "empirically based treatment approaches." My introduction to CBT began in the early '80s. Since that time I have had the incredibly good fortune to meet many of the researchers, educators and thought leaders in psychology through several professional organizations including the American Nurses Association, the Association for Behavioral and Cognitive Therapy, the International Association for Cognitive Psychotherapy, the Academy of Cognitive Therapy and many more. Most recently I attended a large congress on Psychopharmacology coordinated by the Neuroscience Education Institute (NEI), under Stephen Stahl, MD. Dr. Stahl is an accepted world authority on Psychopharmacology. At the NEI Congress, CBT was repeatedly brought up as a solid treatment choice for many disorders either as first line, as an adjunct or as an alternative. Dr. Stahl's team of faculty used brain imaging scans to demonstrate the efficacy of cognitive and behavioral therapies on brain structures as well as on behavioral/cognitive outcomes. Very impressive indeed!

The more we learn about the brain, the more we need to focus on the neurobiological impact of our therapies. It is truly a blessing, as a clinician, to have the option available to use pharmacotherapy, psychotherapy or a combination of the two. I see rapid, exciting changes in patient symptoms and function.

The bottom line is that CBT is effective, well researched as Dr. Leahy as eloquently pointed out, when used by clinicians trained in the model. Perhaps Dr. Allen has had the misfortune of being exposed to one, or more, of the clinicians claiming to be a Cognitive Therapist because they use a few of the techniques as opposed to the full model?

Finally, I hope that opinions that are not science based, as in many of Dr. Allen's comments, do not prevent access to effective care by individuals that could definitely benefit from their use.

First of all I want to thank Dr. Leahy for his thoughtful, succinct and balanced response to Dr. Allen's thoughts. I find these type of discussions useful in that they provoke thoughtful responses on my part with regard to "best practice" and "empirically based treatment approaches." My introduction to CBT began in the early '80s. Since that time I have had the incredibly good fortune to meet many of the researchers, educators and thought leaders in psychology through several professional organizations including the American Nurses Association, the Association for Behavioral and Cognitive Therapy, the International Association for Cognitive Psychotherapy, the Academy of Cognitive Therapy and many more. Most recently I attended a large congress on Psychopharmacology coordinated by the Neuroscience Education Institute (NEI), under Stephen Stahl, MD. Dr. Stahl is an accepted world authority on Psychopharmacology. At the NEI Congress, CBT was repeatedly brought up as a solid treatment choice for many disorders either as first line, as an adjunct or as an alternative. Dr. Stahl's team of faculty used brain imaging scans to demonstrate the efficacy of cognitive and behavioral therapies on brain structures as well as on behavioral/cognitive outcomes. Very impressive indeed!

The more we learn about the brain, the more we need to focus on the neurobiological impact of our therapies. It is truly a blessing, as a clinician, to have the option available to use pharmacotherapy, psychotherapy or a combination of the two. I see rapid, exciting changes in patient symptoms and function.

The bottom line is that CBT is effective, well researched as Dr. Leahy as eloquently pointed out, when used by clinicians trained in the model. Perhaps Dr. Allen has had the misfortune of being exposed to one, or more, of the clinicians claiming to be a Cognitive Therapist because they use a few of the techniques as opposed to the full model?

Finally, I hope that opinions that are not science based, as in many of Dr. Allen's comments, do not prevent access to effective care by individuals that could definitely benefit from their use.

I can understand Dr. Allens point of view. What he is doing is not discarding CBT, as it is understood, apparently, but showing weak points were more and clearer research is needed. He also is neither defending psychoanalysis, nor stating that empirical research is worthless. As I understand it, he states that more research is needed and that research should also include theories that come from psychoanalysis and humanistic approaches, as good as possible.

My point of view is very similar. Psychotherapy is split up in several schools that fight each other instead of working together to come to a integrative, conclusive form of psychotherapy that actually helps people as fast, as effective, and as sustainable as possible. But no. We have the cognitive behavioral, psychodynamic, and humanistic approaches, all doing their own thing, while sometimes hitting each other in the face.

All of us are academics, all of us are grown up men. Can't we just acknowledge one another and work together as a team, instead of taking every little criticism personally? I mean, sure there is a lot of research to be done in psychodynamics, because many psychodynamic therapists just wanted to treat people instead of validating their approaches, but the research keeps coming and should be reinforced, instead shattering of any little claim of anybody going new ways. It's 18th century all over again.

For example. I read a new study, with an 18 month follow up, that shows, that the psychodynamic technique "mentalizing" has an effect-size of 3.1 in treating BPD, while CBT has an effect-size of "only" ~1.4. It's only one study and there is much more to be done to test to scrutiny, especially because -as we know- every study creates more questions than it answers. But it shows, that there are ideas in psychodynamic theories that MIGHT be worth considering.

So WHY exactly do we fight each other? There are many studies that show the effectiveness of CBT, because there are just many studies about CBT. There are not so many studies that show the effectiveness of psychodynamic theories, because there are few psychodynamic studies in general. So what we should do is test specific claims, combine theories, combine what we know about the human psyche, go to the labs and test every single idea we can come up with.

Finally, you used the argument "I want to provide the best possible treatment we have". But don't you want it to be even better? The population of psychiatric patients grows, relapse rates are high, we should think about ways to treat patients more effectively, to minimize mental disorders. THIS is where the focus should be. It's not about who's right or wrong. It's about the patient and how can he be helped as effectively as possible, is it not? And, personally, I'm still not satisfied with what CBT alone has to offer. I think treatment can be improved. So let's get some studies done.

I also would like to thank Dr. Leahy for his response. What strikes me here is that people forget that this is not an academics-only conference, this is the Psychology Today website. The audience is broad - academics, practitioners, lay people, and a lot of folks who may be searching for help because they are in deep pain. We can argue the merits of why some approaches are forgotten and why some have been shown to be more effective, and why we should all just get along until we are blue in the face. Regardless what side you are on, you should ask yourself if someone would be more or less likely to seek any type of treatment at all after reading what you write. I believe that clinicians have a responsibility seek out and deliver accurate information to the public, understanding that they are stewards of treatment and of the profession. If people think that professionals are just hotheads preoccupied with their own agenda and using controversial or inflammatory statements to 'stick it to the competition,' the public will write all of us off. And not trust this field. This can be a barrier to treatment is and a misuse of privilege.

This is why I appreciate Dr. Leahy's response in that it counterbalances Dr. Allen's claims succinctly and professionally. I believe it contributes to the quality of information out there that can be used by people to make their own informed decisions on what approach they would like to try. And the ending was lovely.

I have been reading a number of outcome studies recently because I am seriously worried by claims that 6 to 20 sessions of cognitive behavioural therapy are sufficient to cure such disorders as major depression and anorexia nervosa.

Allow me to summarize, briefly, the findings of a meta-analysis of CBT for bulimia nervosa.

The rate of recovery for patients who completed treatment was found to be around 45%. This is quite substantial - a substantial minority of patients recover after and average of 12 sessions of CBT or behaviour therapy (they are equivalent in effect). It should be noted that there is very little follow-up data by which to judge whether or not these patients remained well.

However, consider the following:

20% of patients dropped out of treatment.

40% of patients who were initially considered for inclusion in the studies were excluded from treatment. This is because, as Dr Allen correctly noted, such studies exclude co-morbid patients (those with multiple diagnoses). Thus, the treatment samples are composed of less complex cases.

As an aside, most outcome studies of CBT for depression exclude around 60-70% of patients - again, because these cases are considered too complex to treat with CBT.

Back to bulimia. On average, after completing treatment, patients continued to binge/purge twice per week. So, although the treatment resulted in a statistically significant reduction in symptoms, many - perhaps most - patients remained symptomatic.

Thus, 45% of a restricted sample (which excluded severely disturbed patients bulimia, patients with bulimia and drug or alcohol addictions, suicidal patients with bulimia and patients with 'borderline personality' disorder and bulimia) reportedly recovered (with little follow up data to support this conjecture).

One of the authors of a study reporting these results concluded that CBT is the "treatment of choice" for Bulimia Nervosa. It is the only treatment that has been adequately studied. This is what Dr. Allen is referring to when he notes that the credentials of CBT are exagerated.

If we actually think about Bulimia in the real world - where most patients have severe co-morbid disorders, and 50% also have a borderline pattern of symptomology - these studies tell us little about the efficicy of CBT. In the lingo of researchers, outcome studies have little 'external validity'.

Why is it that researchers are unwilling to apply CBT to complex or co-modbid cases? They claim it is because they want to exercise experimental control - they want their studies to have internal validity. That is, they want to know which treatment works for which disorder.

It is also very likely that, were researchers to attempt to treat severely disturbed patients with CBT, they would fail to obtain results which reflect well on CBT. They also would have a hard time getting their work published, for journals do not like to publish null-findings.

If one is willing to read the research carefully, and has a basic education in statistics and research methods, the evidence supporting the effectiveness of CBT is very modest.

Indeed, CBT contains a smaller and less diverse 'evidence base' than does cotemporary psychoanalytic psychotherapy.

What CBT has more of than other psychotherapies is outcome research. However essential outcome studies are, they "prove" nothing about the validity CBT. For all they show, the patient might be cured because of a placebo effect or because of cognitive restructuring. Same same but different.

This is called, by the way, the do-do bird effect: the finding that all treatments are equivalent (whether they be behavioural therapy, CBT, 'psychodynamic' therapy, interpersonal therapy and so on). That's what outcome studies tell us. And we don't know why. It seems that the debate is only just starting, and some have already declared CBT the winner.

Thanks, Philip, for taking the time to illustrate what I was talking about. I was too lazy to go back and find examples, although I'm sure the CBT folks will cherry pick a counterexample and claim victory. A great number of their studies are just like you describe.

Even the improvement in Linehan's studies of parasuiciality in borderline patients didn't hold up after two years in one of her studies.

Your post is truly evidence-based.

My pleasure! My concern is mostly about the dominance of CBT - this assumption that there are no viable alternatives. I'm from Australia and the Psychology curriculum here heavily favors CBT - or 'evidence-based' practice.

The most irritating thing is that most Australian psych students are not taught contemporary psychotherapy theory but rather, classical psychoanalysis (as in Freudian metatheory). The tone of the lectures i've had on Freud is either condescending or - very occasionally - overly supportive of his theory. I don't know which is worse - that students are being primed to dismiss psychoanalysis or that we are led to believe that psychoanalysis is classical analysis. This last view is emphatically wrong. What about object relations theory? what about ego psychology? what about self psychology? There is a mass of research behind self psychology in particular.

I do believe that short term therapies can be very effective - broadly speaking, I believe that CBT is effective in the treatment of some anxiety-related disorders and phobias, as well as being an adjunct to counselling practice. I think the evidence will bear this out (eventually). As an example, imagine what a terrible effect chronic and severe anxiety has on a person's life and her ability to learn... given that we know that there is a positive feedback loop between anxiety and arousal, it is often appropriate to intervene with behavior techniques such as PMR (which reduces arousal, thereby reducing anxiety).

I'm less confident about the idea of 'cognitive restructuring' - it seems to me that this sort of thing happens naturally following a reduction of symptoms. The claim that thoughts are causally related to feeling (affect) and behavior is a very very tenuous one indeed, or one that requires a million or so qualifying statements. From my reading, this causal sequence is more mythological than 'empirically supported', and even when it is evidenced, who is to say that disputing negative thoughts and working to re-evaluate one's appraisal of a situation situation will fundamentally change the person? I really doubtful that humans/cognition/affect/motivation/behavior and so on are that simple.

Anyway, back to my original 'beef' with CBT: It is totally baseless for proponents of this technology (or cluster of intervention strategies) to assume that it is 'the' first line treatment for any but a very select number of problems. It is wrong of them to exaggerate findings from outcome studies, just as it is intellectually dishonest of them to over-extend the theory and practice of CBT by using it to treat severely disturbed clients, or clients who have complex problems which will inevitably take a long time to resolve.

That's my beef.

This blog has generated some very worthwhile discussion. Here are some additional thoughts:

I can agree with Dr. Leahy’s assertion that “If psychiatry or psychology is to be taken seriously it must rely on empirical research” only to the extent that the research is of high quality.

I believe the following is a better analogy of psychotherapy with medicine (at least as regarding research) than the one Dr. Leahy gives: When a physician prescribes a medication to me, I read the package insert, because I realize that the physician doesn’t have the time (and probably doesn’t even have the background) to read and critique the research on each drug – and that he/she probably gets most drug information from drug company representatives. Therapy clients (and therapists) need an analogue of a package insert for psychological treatments. It should include information such as whether or not there have been good quality clinical trials of the methodology; indications and contraindications for using it; information such as Philip included in his November 26 comment (e.g., criteria for including participants in the trial; success rate for clients completing treatment, percent of participants who dropped out, types of patients excluded from studies), and probably other information as well (e.g., how improvement was measured).

Since I’m fortunate to have the background to read and critique scientific research, I opted to read the most recent reference that Dr. Leahy gave (Tolin, D.F., Is cognitive-behavioral therapy more effective than other therapies? A meta-analytic review, Clinical Psychology Review 30, 2010, 710 – 720), to judge for myself the quality of the evidence shown in this meta-analysis.

My conclusion: Tolin’s paper takes seriously many criticisms of psychotherapy research, but still needs to address others. If I were grading his paper “on a curve” with the “class” being research in psychotherapy, it would warrant an A. But if I were grading it on the basis of best practices in science, it would rate only a C-. However, I see the positive qualities of the paper as evidence that research in the field is improving. I am optimistic, based this paper, that Tolin can do better on the weak points in this paper in his future research. Regrettably, Leahy’s assertions in his blog are stronger than what is supported by Tolin’s paper.

Some specific comments on Tolin’s paper (please read at least the first; some of the later ones may be too technical for some readers):

The strongest aspects of the paper are:
1. A careful effort to exclude poor-quality studies from the meta-analysis, and especially to include only clinical trials which compared two treatments that were bona fide, and to exclude studies that included an “intent to fail” treatment (for example, restricting therapists in one treatment from engaging in activities that were standard practice in both treatments being studied). The original computerized literature search produced a pool of 219 potential articles for inclusion in the meta-analysis. This pool was reduced to 79 articles by omitting articles that did not provide enough information to perform the meta-analysis, or that did not compare a CBT therapy with a non-CBT therapy, or to avoid duplicating studies. Next, fifty-one (almost two-thirds) of these 79 articles were excluded from the meta-analysis because one of the treatments compared in the article was not bona fide. These figures illustrate how much poor research there is (regrettably) in the field.
2. “Allegiance” was accounted for in the analysis. Tolin defined allegiance (pp. 712 – 713) as “the extent to which you believe in a treatment, expect it to succeed beyond the nonspecific effects of treatment, or identify yourself as a proponent of the treatment.” Since psychotherapy clinical trials cannot be completely blinded (therapists need to know what treatment they are using!), allegiance could plausibly influence outcome. Allegiance was examined for principal investigator, therapists giving treatment, and research team.
3. Tolin started his discussion section with a discussion of the limitations of the analyses in the paper.
4. Conservative assumptions were made when information was available only as a range but exact figures were needed to be included in the meta-analysis calculations.

The weakest aspects of the paper (in addition to the limitations Tolin discussed):
1. (Things start getting more technical here.) Failure to account for multiple testing. When multiple hypothesis tests are performed using the same data, the chance of falsely rejecting a null hypothesis increases with the number of tests. I counted over forty hypothesis tests in Tolin’s paper. This means the chances of having at least one spurious conclusion are quite large. Although the problem of multiple testing has been known at least since the 1940’s, and many ways of dealing with it have been devised, taking it into account is not as widespread in psychology research as in other areas of scientific research using statistics. So it does not surprise me (but still disappoints me) that Tolin neglected this problem. (For more information and references, see J. P. Shaffer, Multiple Hypothesis Testing, Annual Review of Psychology 46, 1995, DOI: 10.1146/annurev.ps.46.020195.003021)
2. Neglecting to give justification for the statistical methods used. The validity of any statistical method depends on certain assumptions (“model assumptions”) that vary from technique to technique. Good statistical practice requires giving thought to whether or not these assumptions are satisfied well enough to warrant using a particular technique. This is not easy, and is often neglected in many papers using statistics. Independence of observations is usually one of the most important assumptions, and is often not present in a meta-analysis. (For example, studies by the same research team are apt not to be independent.)
3. Considering only benefits without discussing possible risks. FDA approval of a drug is based on balancing benefits and risks. Psychotherapy treatments can have negative effects as well as positive effects. (Scott Lilienfeld , Psychological Treatments That Cause Harm, Perspectives on Psychological Science March 2007 2: 53-70, gives a good discussion of this.) Tolin’s paper does not discuss any negative effects of CBT, but his abstract ends with the statement that the results of the paper “suggest that CBT should be considered a first-line psychosocial treatment of choice, at least for patients with anxiety and depressive disorders.” [Disclosure: I have experienced negative effects in psychotherapy. However, I also tend to have a bias in favor of some types of behavioral methods.]

Aspects of the paper that display both some strengths and some weaknesses:
1. Handling of intent-to-treat analysis. In analyzing results of clinical trials, it is important to base the analysis on all participants who were randomized to treatment, not just those who completed the treatment (since the latter destroys the randomization, as well as being of little relevance in choosing a treatment). This is called intent-to-treat analysis. Although Tolin mentions intent-to-treat analysis, it is not clear to me from the paper whether he included only intent-to-treat analyses in his meta-analysis, or whether he included studies in which the only analysis given was based on those who completed treatment.
2. Handling of measures of effectiveness. The limitations did discuss quality of measures of effectiveness, but conclusions were based only on statistical significance. It is important also to consider “clinical significance” (also called “practical significance”). For example, if average differences in effectiveness between treatments are so small that they are within the normal range of variability in how a person might answer from one day to the next, then that difference is too small to conclude a difference in the treatment effects, even if the difference is statistically significant. Tolin did not discuss clinical significance.