There Are Better Ways to Judge People Than by Their Faces

How well can you judge the honesty of a person you’ve just met? Do you think there are tell-tale signs of trouble that may emerge once you get to know this person better? Perhaps you focus on their eyes, peering into what you believe are the depths of their soul, if not character. When you’re in the middle of a crowd and need some help holding onto a slippery package, you may be convinced that you can pick out that one person who will give a hand, not swipe the package away from you.

As noted in their introduction to a new and comprehensive meta-analysis of facial judgments of honesty, the University of Western Australia’s Yong Zhi Foo and colleagues (2022): “Facial impressions could also simply represent perceptual overgeneralizations based on other, more valid cues, such as those provided by emotion, facial maturity, or sex” (p. 1580). In other words, they may be completely off base.

Why, then, do people have so much confidence in the importance of the face as a signifier of what’s going on behind the scenes? Indeed, the authors go on to observe that “Overall, however, it is puzzling to explain why perceivers form these judgments so readily, if they are wholly inaccurate.”

Two Ways to Gauge Facial Judgment Accuracy

When examining the (in)accuracy of facial judgments, there are two ways to go about the process. First is the “face level” analysis, in which it’s possible to calculate a score corresponding to someone’s trustworthiness as judged by experimental raters. Your face may rate a “5” on a 10-point scale, and someone else’s a “3.”

Then, to validate this score, researchers would have you complete a lab test in which your trustworthiness would actually be measured (such as whether you lie on a coin-flip task). This type of rating falls in the category of “ground-truth” trustworthiness which the authors note falls in the category of the wisdom of the crowd.

The second measure of facial judgment accuracy is based on the perceiver’s own characteristics and ability to pick out someone known to be untrustworthy. In this case, if you’re high on this quality, you’ll be able to select the would-be helper in the crowd. Studies based on perceiver-level accuracy, therefore, have a different focus than face-level accuracy studies, but both should be able to produce data on face judgment accuracy.

The stakes in these analyses, whether based on target or perceiver, are high. Again, quoting the authors, “It is vital that lawmakers, the media, and the public know to what extent we can rely upon our first impressions of trustworthiness” (p. 1581). Not only is this information helpful for your personal safety, but a wrong judgment could lead an unfairly convicted person to a lifetime of incarceration, all other factors being equal (and, indeed, as claimed by the authors).

What Do the Data Say?

Using the powerful tool of meta-analysis, Foo and his colleagues report in-depth on their methods of securing relevant studies to subject to statistical analysis. They defined “good and bad intentions” as composed of any of the following behaviors: trustworthiness, aggressiveness, agreeableness, criminality, deception/honesty, and sexual unfaithfulness.

As is often the case in studies such as this, the large initial pool readily becomes narrowed to a much smaller set that meet specific inclusion criteria, and as it happened, from the 15,832 initially identified, only 25 provided usable data. Though the final number seems small, the authors actually could analyze 71 effects based on 1,976 unique faces and 3,500 unique participants.

Sex Essential Reads

Not Attracted to Your Partner Anymore?

Can Casual Sex Be a Source of Emotional Relief?

From their systematic analysis of their combined data set the research team arrived at a somewhat underwhelming effect size of .16 (out of a max of 1.00) for face-level studies, statistically significant but small. The authors did observe a slightly higher effect size of .27 for perceiver-level studies. The “kernel of truth” in face-level analyses was therefore mirrored with a slightly larger “kernel” for perceiver-level studies.

Noting that the studies included in this analysis reflected decades' worth of work from multiple countries, the authors then conducted one additional control called “research weaving” which allowed them to trace the time frame and origins of the included studies. This analysis uncovered several significant limitations in the field as a whole, including a lack of consistency among researchers in methods and measures and, importantly, a lack of cross-talk among research labs. There was also a failure for most of the research to include diverse samples, and many participants were college students.

What’s Really Going on in Facial Judgments and What Can You Do?

The kernel of truth finding may not be very comforting to you, then, if your hope was that this far-reaching study could give you some simple guides to use in your own trustworthiness judgments. You definitely would not be comforted by this next problem, which involves the role that facial judgments might play in real-life situations.

As Foo et al. note, the judgments that people make of other people’s faces don’t occur in a vacuum. Let's say that someone has what appears to be a dishonest face according to the wisdom of the crowd. Believing this person to be dishonest, others then treat them in a way that signals their lack of faith in them. As a result, a self-fulfilling prophecy becomes set up and they actually become the untrustworthy person that their face labeled them as being.

In the words of the authors, “Over time, these behaviors could become self-reinforcing, especially as people appear to be aware of how trustworthy they themselves appear and use this knowledge strategically” (p. 1591). Here, then, is the ultimate irony. Being told that they have an “honest face,” these individuals could then feel that they have the license to misbehave.

You know now that you can’t trust your judgments, or at least only to a very minimal degree. So, what’s left? The UWA team notes that all is not lost.

Rather than look at the face as your basis for judgment, try looking instead at the person’s behavior: “Appearance-based judgment is an irrational substitute for evidence-based judgment” (p. 1591). While in that crowd, for example, you could see who seems to be helping others before trusting them to help you. Do they stand aside and let others pass, or do they seem intent on getting where they’re going, come what may?

You can also be somewhat assured by the fact that in real life, you don’t just have a snapshot image to use as the basis for your judgments. You can see people you interact with in multiple contexts, which could help give you more data about whether they deserve your trust.

To sum up, kernel or not, judgments based on facial characteristics alone don’t hold up to scientific scrutiny. To give yourself the best chance of judging people both fairly and accurately, put your trust in what people do, not how they look.