How Do Modern Personality Tests Work?

Millions of people take personality tests either to learn about themselves or because others—like potential employers—want to know about them.

To some, these tests can seem like magical devices that reveal their hidden selves. Others may see them as bogus gimmicks that are useless or even actively deceitful.

But personality tests used and developed by scientists or respectable companies are neither magic nor bogus. Instead, they are transparent and straightforward tools anyone can understand, even though creating them requires training and diligence.

The tests achieve something hard to accomplish otherwise: they systematically ask about many aspects of people’s personalities and directly compare their traits to those of many other people.

A good test is like a good interview

Think of a personality test creator like an interviewer and the test taker as their interviewee. An interview works best if both parties know its purpose, understand the questions and answer them honestly. The same goes for personality tests.

Granted, there have been 'personality tests' trying to trick people into indirectly giving away their secrets. For example, some tests have asked people to tell what they see in ink blobs, supposedly revealing their hidden thoughts and feelings. Other tests have used obscure questions with no apparent relevance for personality, like “My mother was a good woman.” But there is little evidence that such trickery provides much useful information.

Also, some tests have attempted to catch people lying. For example, how could honest people agree with a statement like “I am never unkind to anyone”? But basically honest people often respond affirmatively to questions like these, so this trick to catch liars doesn’t work. It looks like most people complete tests in good faith, not taking such catch-statements literally but rather as strongly-worded attempts to describe actual personality traits.

So, instead of secret service-like interrogation techniques, most modern personality tests ask questions with clear intent and meaning. This way, test takers understand what they are being asked about and can answer accordingly. Test creators and takers are expected to trust each other.

Here’s how a modern test is created

In principle, creating a personality test involves only a few key ideas. But it takes tedious work and appropriate training to get every step right.

• The test creator first defines the trait they want to measure. Suppose they want to measure extraversion: how is its high level expressed? What shows its lower levels?
• They then write many diverse statements (called items) that describe different aspects and levels of the trait. This allows broadly sampling the many behaviors, thoughts, and feelings in which extraversion can manifest. It takes at least ten, but ideally a few dozen items to measure the trait well. Before selecting the items, test creators often trial many more.
• Among others, extraversion items may include: “I find it easy to make friends,” “I often take control in social situations,” or “I usually feel full of energy."
• Test takers rate how well the items describe them, often using a five-point scale from “completely disagree” to “completely agree.” A person’s extraversion score is the average of their responses to all extraversion items.
• Besides representing the trait well, averaging responses to several items accomplishes another trick: many unwanted influences to individual responses—for example, items’ unique meanings or the test taker being distracted—tend to cancel out. Consequently, the result reflects more extraversion and less of anything else.
• Personality traits do not have a minimum and maximum value, so individuals’ test scores can only be meaningfully interpreted in comparison to others’ scores, which are called norms. Just like the items need to represent extraversion thoroughly, the norms need to thoroughly represent the people you want your extraversion to be compared to.

Making sure that the test actually works

To assess the test’s quality, its creators study how similar the scores remain over a short time period and how well they match the scores of other tests measuring this trait. Besides, they may ask that test takers’ personality traits be additionally rated by their friends or partners, expecting the results to be similar regardless of who provides the ratings. I explain this in another post.

Test creators also study whether people’s test scores track other relevant information about them. For example, those higher in extraversion should have more friends, attend more social events and be more likely to be in leadership roles.

Also, because family members tend to be similar in any measurable characteristic, test creators may expect relatives', especially identical twins’, test scores to be more alike than those of strangers.

These are the kinds of evidence a modern personality test can show for its quality. The more such evidence it can boast, the more trustworthy it is.

What you put in is what you get out

Modern tests’ transparency is their great strength. But this also has a downside: the quality of their results depends on test takers’ integrity, which can vary according to the situation. For example, people are often highly motivated to present themselves desirably when applying for a job, but less so when completing the tests to learn about themselves.

Personality Essential Reads

Can Food Change Your Personality?

How What You Eat Reveals Your Personality

Yet personality tests do generally work, even in situations that do not incentivize being entirely honest. For example, test scores somewhat predict future job performance—far from perfectly, but often sufficiently well to be helpful in making hiring decisions.

So, here’s one way to look at personality tests: Instead of worrying too much about their limitations, we can be amazed that such simple, transparent, and cheap-to-use tools work at all.

They work because most test takers and creators have enough trust in each other.