A meta-analysis by Falk Leichsenring and Sven Rabung is an effort to muster support for Leichsenring's campaign to obtain reimbursement of long-term psychodynamic therapy (LTPP) by German insurance companies. Leichsenring has achieved international visibility as a persistent advocate for LTPP and pitbull critic of anyone who questions its efficacy. He serves as representative for the interests of long-term psychodynamic therapy on the Wissenschaftlicher Beirat Psychotherapie (WBP), a body whose judgments about the scientific status of psychotherapies are accepted by German insurance companies as the basis for reimbursement. Currently, LTPP does not receive reimbursement, based on WBP's finding of a lack of evidence for the added expense of longer-term treatment. Leichsenring's agenda in writing the BJP article becomes obvious in the discussion section:

Long-term psychotherapy... is associated with higher direct costs than short-term psychotherapy. For this reason it is important to know whether the effects of long-term psychotherapy exceed those of short-term treatments. In this meta-analysis, LTPP was superior to less intensive methods of psychotherapy in complex mental disorders. Furthermore, we found positive correlations between outcome and duration or dosage of therapy.

In this blog, I consider this conclusion. But, first it is important to note that this article in BJP has a serious overlap with a review Leichsenring and Rabung published  in JAMA. All but one the 10 studies reviewed in the BJP article were included in the previous JAMA paper and the tenth study is irrelevant to the evaluation of LTPP versus other psychotherapies.

The added study compares an 18 month "mentalization-based" therapy to structured clinical management, "a counseling model closest to a supportive approach  with case management, advocacy support, and problem-oriented  psychotherapeutic  interventions (p.357)." This study is not relevant to the evaluation of whether LTPP is superior to other psychotherapies, because it compares LTPP to a mixed assortment of treatments available in a psychiatric setting, not just psychotherapy, with no statistical controls for the other treatments such as medication or case management.

Leichsenring and Rabung cite their JAMA review but failed to alert readers that estimates of the efficacy of LTPP there were exaggerated. In JAMA, Leichsenring and Rabung reported a summary effect size of 6.9 for a set of studies, none of which individually had an effect size of more than 2. So, they combined effects of a set of studies and claim an overall effect more than three times the effect of the study with the largest effect. This is a voodoo statistic. They corrected themselves in the BJP article without acknowledging their initial mistake in JAMA.

Leichsenring and Rabung claimed they rated the quality of the studies, but they mistakenly used a scale for rating the transparency with which a study is reported, not the overall quality. My colleagues and I reviewed the quality of the studies that overlap between the JAMA and BJP papers using Cochrane Collaboration guidelines. All the studies were of poor methodological quality, and none met the important criteria of being low in bias incompleteness of data.  These studies were pitifully small, with no study having more than 31 patients in either treatment or control conditions, with one group having only 15 patients.

Leichsenring and Rabung liberally and selectively interpreted their inclusion criteria. One study that was favorable to LTPP was included despite having been classified as a short-term psychodynamic therapy in another of their meta-analyses.

Contrary to established standards, Leichsenring and Rabung do not provide a list of excluded studies. Their inclusion criteria required LTPP to be at of at least one year or 50 session duration, but there was no such requirement for comparison groups, which could be much shorter or no treatment. Moreover, most clinicians using evidence-based psychotherapies will not commit themselves to requiring one year of treatment and these therapies are almost always evaluated in trials with fewer sessions and less duration. Results of most high quality studies that evaluate evidence based therapies were thus excluded from consideration in the BJP meta analysis. 

Leichsenring and Rabung cherry picked evidence in the studies they included to make LTPP look better. One study actually had better results for the solution-focused control condition at one year follow-up, no differences at two year follow-up, and better outcomes for LTPP at three years. The LTPP was much frequent (2 to 3 times per week versus one session every second or third week) and longer (up to three years versus a maximum of 12 sessions over no more than eight months). Leichsenring and Rabung selected the 3 year follow up for inclusion in the meta-analysis, at which time the LTPP could still be ongoing, but the short-term therapy would have been completed over two years ago.

The comparison groups to which LTPP were compared in the BJP meta-analysis were quite diverse. For one comparison group, the original report indicated that participants received no formal psychotherapy and another involved patients remaining on a wait list. Only two comparison groups were actually evidence-based treatments being applied to conditions for which they were validated. In both instances, the treatment of the comparison groups was equally or more effective than LTPP, despite the treatments being of substantially shorter duration. Leichsenring and Rabung claimed to have examined statistically the effects of the diversity (heterogeneity) of the comparison-control groups, but the statistical approach that they use becomes meaningless when only 10 studies are examined.

The bottom line is that this seriously flawed meta-analysis does not provide accurate information to clinicians or policymakers who wish to consider the efficacy of LTPP relative to other specific treatments.

Leichsenring and Rabung raise but never address the issue of the relative costs of LTPP versus alternative treatments. Brett Thombs and colleagues provided a provocative cost analyses of study comparing LTPP to solution-focused therapy Leichsenring and Rabung reviewed.