MBTI, If You Want Me Back, You Need to Change Too
It's time for the MBTI to embrace the standards of science
Posted Sep 24, 2013
Dear Myers-Briggs Type Indicator,
Do you remember the day we met? I was a wide-eyed high school senior, and you were an exotic beauty. It was love at first sight. Our first date was magical: I opened up to you like I had never done with anyone before. In return, you opened my eyes to a whole new way of seeing the world.
We had so much in common back then. Sadly, as the months passed, we started to grow apart. It began when I met your family.
Your mother and grandmother were obsessed with Carl Jung, who made up his three “types” based on his personal experiences rather than science (with the help of your mother, who made up the fourth). You had years of those experiences, and I was young and naïve, so why would I doubt you? But when I studied for a doctorate of my own, I learned that this was Mesearch, not Research. And a new girl caught my eye. Her name was Big Five, and she was raised by an entire extended family with PhDs in psychology, over multiple generations. They gave birth to her through a very different process. Instead of relying on their own limited experiences, they went out and polled thousands of people in different parts of the world, to find out how they viewed personality.
Instead of inventing categories, Big Five’s ancestors realized that the major dimensions of personality could be found in natural language. If we look across the world’s cultures, we should find words to describe the most important psychological characteristics of people. One study included 1,710 adjectives in English, which ultimately made up five major categories of personality, not four. She was multicultural: the same basic categories replicated in many languages, from Chinese to Filipino, German to Italian, Dutch to Polish, and Hebrew to Russian. They called her Big Five.
Of course, Big Five’s parents realized that language is only one of many ways to see personality. To make sure that their categories were meaningful, they collected genetic evidence and fMRI data. They also found that there was really no such thing as a type—every personality trait was on a continuum, and it was very rare to be on one extreme or another. Type wasn’t the only one of Jung’s original ideas that didn’t pan out. You said extraverts focused on the outer world and introverts on the inner world, but Big Five’s ancestors discovered that this was really about sensitivity to stimulation and social attention. You taught me that most people had a dominant preference for thinking or feeling, but research demonstrates that whether you prefer to use logic when making decisions has nothing to do with whether you’re concerned about how those decisions affect others. Giving me a thinking-feeling score is not like assessing whether I’m right-handed or left-handed. It’s more like evaluating whether I prefer soccer or Swiss cheese.
Research is usually more accurate than Mesearch, so you can imagine my chagrin when you told me you weren’t going to change. You were like the Catholic Church clinging fiercely to the idea that the sun revolved around the earth, even after the discoveries of Copernicus, Galileo, Kepler, and others showed the opposite. One of your family members called Jung’s book her Bible, and you clung to it like it was an actual religion.
I tried valiantly to open your mind. I even sent you a rad mix tape with Big Five’s favorite songs on it! I told you that theories should be revised based on data—if we don’t learn from our research, what’s the point?—but you were blinded by Jung’s armchair theory. I couldn’t take it anymore, and I broke up with you.
In response, you left me a message. When you and I were together, you only knew a part of me. Big Five knows me much better; I’m an open book with her. (That includes my voicemail password, which is why I’m in the doghouse right now—she didn’t know we were back in touch. For the record, she was exploring her future with HEXACO, so I wasn’t cheating on her. WE WERE ON A BREAK!)
The truth is, MBTI, you were my first love. Since then, I’ve changed, and a relationship is a two-way street. If you want me back, you need to change too. All I ask is that you follow a few basic rules of science…
1. Embrace the Latest Data to Improve the Theory
MBTI, darling, you mentioned a team that’s tasked with “keeping the instrument up to date.” Who updates the theory itself? Would it be so terrible to abandon Jung’s creative but antiquated ideas, and start capturing a broader set of preferences? If that’s too much to ask, and you’re devoted to your categories, why do you still insist on calling them types, when we know traits are more reliable and accurate? As Pittenger advised in his review, “those interested in using the MBTI should examine the advantages of replacing the four-letter type formula with more traditional magnitude assessments of personality.”
Also, how about amending your thinking-feeling scores to separate these two dimensions? Your current flame acknowledges that “The T-F scale tends to have the lowest reliability,” so why not give it a tune-up? While you’re at it, it would mean a lot to me if you refined your definition of extraversion to reflect all that we’ve learned from half a century of systematic research. Some of your cousins have been perfectly comfortable with all of these ideas for a while. A quarter century ago, Cowan (an MBTI proponent) wrote, “there is no obvious reason why the current status quo of this theory and its measurement cannot be improved.”
2. Give Us Real Evidence for Efficacy
You told me that if the MBTI “lacked a solid research-based foundation, it wouldn’t be used by the world’s top organizations, including Hallmark Cards and Southwest Airlines.” Since when do “the world’s top organizations” have the capabilities to evaluate the quality of evidence for a practice? Most of the world’s top organizations continue to use unstructured interviews, even though evidence shows that they’re highly ineffective. Many use forced-rankings to evaluate employees, despite a lack of consistent or convincing evidence that they work. Acceptance by non-experts isn’t a marker of validity. It’s a signal of popularity.
You said “its efficacy is so well-established,” but your supporting evidence for that efficacy was a series of case studies. Experts in medicine and management agree that case studies are exceptionally weak forms of evidence. In science, the best evidence comes from meta-analyses of randomized controlled trials. To validate the “efficacy” of the MBTI, you need randomized, controlled experiments demonstrating that participants gain more insight from the MBTI than other comparable tools. I’m willing to bet that if you take the Big Five or HEXACO and walk participants through the same process—provided that the trainers are equally zealous—the MBTI will fare no better. But I’m also willing to be wrong, because I am committed to revising my beliefs based on high-quality evidence. (I say this at the risk of you labeling me as a thinker-perceiver-senser. I know you don’t believe that some preferences are inherently better than others, but surely science requires some thinking and perceiving and sensing. Otherwise, my dear, we will have no analysis of facts and updating of knowledge.)
After I dumped you, one of your relatives, Hile Rutledge, weighed in on our breakup. He has some perspective to offer, since he’s partially familiar with our history, and he’s been in an open relationship with you and Big Five. I happened to agree with most of his thought-provoking comments. There was one point, though, that puzzled me. He called you a “client-centered tool that builds self-awareness and helps lead to better self-management and growth.” To paraphrase my buddy Jerry, show me the data! Where are the experiments documenting that (a) taking the MBTI leads to self-awareness, self-management, and growth, (b) these benefits can emerge even in the absence of a zealous trainer or coach, and (c) it offers insight that exceeds, or at least matches, other psychological assessments?
Also, can you show us in a randomized, controlled trial that the “insight” you describe is, as you promised, “extremely beneficial… in enabling groups of people of varying personality preferences to work together cohesively”? If you believe that insight leads to cohesion when working with people of different preferences, why haven’t you conducted experiments to test whether cohesion rises when people learn about their “types”? And since there is already extensive evidence that cohesive groups tend to perform more effectively, why can’t you link this anticipated cohesion benefit to group effectiveness?
You said “it is true that the MBTI instrument does not predict performance or satisfaction within an occupation.” Big Five has no trouble with this. Now, you may tell me that you’re not interested in Big Five’s way of looking at the world. But you do have one relative in common, and that’s introversion-extraversion. Consider this evidence from Big Five’s extraversion measure:
- Teams with a mix of introverts and extraverts tend to outperform those with mostly extraverts or mostly introverts.
- Extraverts sell more when they’re directly rewarded, and ambiverts bring in more revenue than introverts or extraverts.
- Extraverted leadership is linked to higher unit profits when employees are passive but lower profits when employees are proactive.
Big Five didn’t just give me insight; she helped me figure out where I was most likely to be satisfied and effective. Why doesn’t your extraversion measure do that? I want you to be a better person, MBTI, and it makes me sad that you don’t seem to care about the happiness or success of your lovers. Can we work on your feeling score, just a little bit?
3. Hello Pot, I Am Kettle
I appreciated that—unlike many of your current partners—you resisted the temptation to diagnose my “type.” I did, however, take issue with a few of your jabs:
(a) Whose Research Diet is Lacking Iron?
You called my sources “anemic.” One of those sources included a comprehensive review of all peer-reviewed studies published on the MBTI in management. The authors seem to be favorably disposed toward you, but they still wrote: “Our critique of the management type studies identified many weaknesses… many findings are suspect because they are inconsistent across studies and/ or weak.” They go on to make a strong recommendation for “more rigorous research designs.”
(b) Unreliable Arguments about Reliability
You told me that “The test-retest correlations for the most recent version of the Myers-Briggs tool are in the range of .57 to .81, which is considered quite good for psychometric assessments.”
That’s not my read of psychometrics. A test-retest correlation of .57 means that the shared variance between the two tests is only 32%. Setting statistical cutoffs is always arbitrary, but as far back as 1978, the esteemed psychometrician Jum Nunnally recommended a minimum of .70. But MBTI, you know as well as I do that the gold standard for construct validation is to separate trait and method variance with a multi-trait multi-method matrix. Do you have one yet? The only one that I could locate excluded J-P, and showed that “reliability for TF was not sufficient.”
You celebrated your 50th birthday last year—don’t you think it’s time that you had those warts removed from half of your body?
(c) Convoluted Claims
You wrote: “Grant asks why the MBTI assessment remains popular, and poses several convoluted explanations.”
Here were my explanations, verbatim: “people cling to the test for two major reasons. One is that thousands of people have invested time and money in becoming MBTI-certified trainers and coaches. As I wrote over the summer, it’s awfully hard to let go of our big commitments. The other is the “aha” moment that people experience when the test gives them insight about others—and especially themselves.”
I am genuinely confused here. I know you don’t usually pay any attention to other people’s ideas or their data, but can you help me understand what’s convoluted about these two explanations? Both are based on extensive research—the first comes from studies of escalation of commitment to losing courses of action, and the second comes from studies of the Forer effect (also known as the Barnum effect).
I Remember the First Time We Broke Up
MBTI, this isn’t just about you and me. You see, this week, many of your exes have come out of the woodwork. They happen to be scientists, and they all left you for the same reasons that I did.
One of the virtues of research is that we can agree on the standards for a fair experiment. In fact, when goal-setting experts disagreed vehemently about whether goals needed to be chosen or could be assigned, the chief antagonists resolved the debate by collaborating on the design of a joint experiment. If you’re willing to embark on such an experiment with one of your antagonists, I’ll take it as a sign that you’re no tool—you might be ready for a committed relationship.
If not, we are never, ever, ever getting back together.
Adam is an organizational psychologist and the author of Give and Take: A Revolutionary Approach to Success, a New York Times and Wall Street Journal bestseller. Follow him on LinkedIn at www.linkedin.com/influencer/profadamgrant and on Twitter @AdamMGrant