Hooked on Patterns

Demystifying statistics, machine learning, and “Big Data” in mental health and substance abuse

12 Steps to Abstinence? Not for Everyone

Revisiting patient-treatment matching in treating alcohol use disorders

Comparing CBT, MET and TSF
Comparison of Cognitive Behavioral Therapy, Motivational Enhancement Therapy and Twelve Step Facilitation Reveals NO DIFFERENCE
Journal of Studies on Alcohol, 58(1), 7-29.

When we think of alcoholism, we think of Alcoholics Anonymous (AA): the rituals, the sponsors, the celebrity interventions, the “I am an alcoholic” retort, and the lives that it saves. Without question, AA has been an effective treatment for many patients, and the group support model has been broadly disseminated. But despite what many have been led to believe, it isn’t the only effective treatment for alcoholism, and certainly isn’t the only effective treatment for everyone. Lately, popular media has become enamored with these “alternative” psychotherapies and medications. Debunking AA is made fashionable.

In fact, these treatments are not “alternative” at all—or even particularly new. Treatment such as Cognitive Behavioral Therapy (CBT) and naltrexone have been around for a long time, but many patients and clinicians are either unaware of or resist their use. The reason is two fold. First, while CBT and psychotropic medications are the mainstay in general psychiatry, substance use disorders treatment has been historically segregated from mental health care. Substance using patients resist being labeled “mentally ill” and taking advantage of medications and behaviorally driven therapies. Secondly, while we know that many treatment strategies work on average, we do not know for whom they work, and this unpredictability on an individual basis makes it hard to convince both patients and doctors that something new is a better, or even a valid, alternative. Treatment decisions are made primarily based on culture and tradition, rather than on scientific evidence.

Everyone knows that patient-treatment matching is the holy grail. This old question re-branded itself as the sexy new field of “personalized medicine” or “precision medicine,” but research along those lines has existed since the '80s. One of the biggest and most expensive randomized clinical trial publicly sponsored by The National Institutes of Health (NIH) and National Institute on Alcohol Abuse and Alcoholism (NIAAA) in all of behavioral health is called Project MATCH, which was started in 1989 and examined exactly this question: Cognitive Behavioral Therapy, Motivational Enhancement Therapy (MET) and 12-Step Facilitation (i.e. AA), which one is the most effective? And for whom is it most effective? The trial randomized 1726 patients in both inpatient and outpatient settings and cost more than 30 million dollars. Many papers and books have since come out of the study, but as an aside, the number of citations of the primary outcome paper [1] is not nearly as numerous as I would have expected for such a pivotal trial.

Here is the clincher: all of these treatments are about equally effective on average (Figure 1). Keep in mind, Cognitive Behavioral Therapy is a once weekly treatment for about two to three months and Motivational Enhancement Therapy is only a few sessions altogether. The intensity for 12-Step Facilitation, on the other hand, is more variable: patients can attend an AA meeting every day, or they can attend no AA meeting at all. Numerous studies have examined clinical predictors and correlations to many different outcomes, and their interactions with treatment. Disappointingly, no clinically useful predictor has emerged, and to this day we still do not know which individuals would benefit more from 12-Step Facilitation versus Cognitive Behavioral Therapy.

The blind continue to lead the blind. 

Why did Project MATCH fail to find what it was designed to find? Many very smart people have proposed many interesting theories. Some believe that there are fundamental issues in the trial design. Others believe that these existing treatments simply do not work well, and people primarily recover on their own. The debate is ongoing and at times heated. Even though I am not an expert in this area and do not want to take sides, as a 21st century methodologist, I am still very much impressed by the statistical rigor in this trial.

My theory on Project MATCH is somewhat different and informed by the more recent results from Genome Wide Association Studies (GWAS). For the aficionados, GWAS are studies in which a single genetic variation is correlated to a clinical outcome, such as alcoholism. In a GWAS, many hundreds of thousands of these single genetic variations (“loci”) are tested for achieving statistical significance in such a correlation, but often few, if any, achieves the significance threshold. Even when we do identify a handful of these significant genetic variations, we still cannot predict who will grow taller, or more intelligent, or more beautiful. People’s genomes are more than a bag of individual variations: each variation only exerts a small influence on the final outcome. Using genetic information should be more like reading a signature than finding a needle in a haystack.

My conjecture is that the reason we cannot predict who will be a good candidate for a 12-Step program versus Cognitive Behavioral Therapy is exactly the same. Each potential predictor—your personality, your upbringing, your beliefs and style—contributes only a small amount to the final outcome of treatment, and the predictors interact in complex ways. Fishing for statistical significance, on a predictor by predictor basis, is a failed proposition from the start. We have to combine all these predictors in their entirety—let’s call it the “psychosociomics” signature—in some rational way. And we do not yet know how to do that.

Is this the end? It almost looked like it in the late '90s. In a very short period of time in the early 21st century, however, a set of powerful algorithms emerged from machine learning to essentially read and classify complex signatures from many different areas of science. These techniques have been used to build the Google car and read your handwriting in an ATM machine, but they have not been well modified and optimized to extract useful patterns from clinical trials, and certainly nobody has looked at applying them on Project MATCH—an old, unsexy, out of fashion dataset that is collecting dust in the attic of time. In my mind’s eye, that “signature for a perfect candidate for AA” is buried somewhere in there.

This needs to change, and I am optimistic that it will. NIAAA and National Institute of Drug Abuse (NIDA) have announced a new funding mechanism last year to encourage substantial work in the secondary analyses of old, large, multi-center trial data. Someone needs to write a grant to do this work so that one day when we have a patient in the clinic we can pull up an app, punch in the numbers, and be able to tell him or her, “it looks like you should go to AA. Really really.”


[1] Project Match Research Group. (1997). Matching alcoholism treatments to client heterogeneity: Project MATCH Posttreatment drinking outcomes. 

Sean X. Luo, M.D., Ph.D. is a physician-scientist working at Columbia University and The New York State Psychiatric Institute.

Subscribe to Hooked on Patterns

Current Issue

Love & Lust

Who says marriage is where desire goes to die?