The Dress, 10 Years On

Before the coronavirus pandemic, before the conflict in Ukraine, before chatGPT, there was #TheDress. Strangely, it feels at once like it happened in a different era — really a lifetime ago — but also like it was just yesterday.

We were all there, we all remember it: An image of a dress, shared online, sparked intense debate, as some clearly perceived it as white and gold, while others were certain it was black and blue—even when viewing it on the same screen, so differences in perception could not be attributed to different brightness settings. These were not tranquil times in many households.

This is not about the dress itself, which is black and blue, but its picture. In this picture, the lighting conditions are unclear — the dress could conceivably be illuminated by daylight or artificial light.

Why should this matter? Because our visual system — when using light reflected from objects to infer their material properties — must solve an inverse problem. To illustrate what an inverse problem is, consider the case of a light object (reflecting most of the light) and that of a dark object (absorbing most of the light):

6–1=5

9–4=5

The first number represents the strength of the illumination and the second how much of the light is absorbed by the object, the third number how much light falls on the retina. How would the brain know if a bright object is illuminated by a dim light, or a dark object by a bright light? To the eye, they both look the same, so a reliable inference is impossible.

Fortunately for us, the brain solves this problem routinely with a color-correcting mechanism called “color constancy”. If the source of the light is clear, it is straightforward to unconsciously account for it. This was important even in times before artificial lighting, as noonish daylight has a dramatically different wavelength mix than the warmer hues of the evening sun. When the sun is lower in the sky, sunlight travels through more of Earth’s atmosphere, scattering shorter — blueish — wavelengths, a fact well known to all photographers who use this “golden hour” to great advantage.

But what does any of this have to do with the dress and how it is perceived? As we discussed, the lighting conditions in the image are ambiguous. So here, color constancy operates on one’s interpretation of the light source. Thus, individuals who interpret the dress as being lit by bluish daylight are more likely to perceive it as white and gold, whereas those who assume a yellowish artificial light source tend to see it as black and blue.

Assumptions are shaped by prior experiences; we assume what is typically true. So, this boils down to how much daylight someone was exposed to, relative to artificial light. The higher this ratio, the more likely they would see the dress as white and gold. Measuring someone's lifetime exposure to natural versus artificial light is impractical, but chronotype offers a useful proxy: People who stay up late are more exposed to artificial lighting and might be more inclined to see the dress as black and blue, whereas early risers — spending more time in daylight — are more likely to perceive it as white and gold. Empirical results are consistent with this prediction.

Experience with light—the “prior” in the parlance of our times—is what matters to determine the percept of the dress. It’s as if everyone was wearing goggles that are tinted by their past. We literally see the world differently, depending on our own experience.

Of course, an after-the-fact explanation of an idiosyncratic image is at best a starting point for science. If it is true that the differential interpretation of the light source causes the disagreement about the percept, we should be able to recreate the effect de-novo:

And we did: We put a pink croc under green light so it looks grey, then added white socks which — reflecting the green light appeared green. People who know that these socks are white used the green tint as a cue that something is off with the light and mentally color-corrected the image. To them, the croc looked pink, even though the pixels are objectively grey. People who took the color of the socks — green — at face value, saw the croc — consistent with its pixel values – as grey.

The pink crocs and white socks under green light

Source: Pascal Wallisch

We call this mechanism “SURFPAD” (Substantial Uncertainty & Ramified/Forked Priors/Assumptions = Disagreement).

We suspect that this principle has ramifications far beyond the dress and internet memes.

Note that when information about the illumination source was lacking, the color constancy mechanism was not disabled. Instead, it was operating on a guess. It was an educated guess — based on past information — but a guess nonetheless. Your brain is not telling you that it was guessing, because — much like the chatbot — it is always guessing. We are not consciously aware that priors were used in generating the percept. These priors are a statistical mélange of what we experienced in the past, much like chatGPT uses the statistics of its training data to generate responses. Doing so, hallucinations are inevitable. Similarly, much of what we see seems to be informed by information that is not actually there right now. Simply put: Informed hallucinating, particularly if the information available is ambiguous.

Memes Essential Reads

How Communication Changes in a Post-Literate Society

Why Crypto Memes Hijack Your Brain, and How to Resist

Why does this matter?

Say two people support different political parties. One person read in the newspaper that a politician did something bad, which led them to conclude that this politician is a bad person. Neither knows the politician, so the other person points out that the news has the power to put anyone in a bad light, so all this shows is that the light is biased. Remarkably, both sides will feel vindicated — the first person will interpret this as evidence that the politician is bad, because they look bad. The second will see this as further evidence of media bias and corruption. Remarkably, both will be strengthened in their convictions, an apparent confirmation of their beliefs. And so, they will drift ever farther apart.

Of course, everyone knows that a house divided against itself cannot stand.

So, what will we do?

Respect priors!

PS: If you want to participate in our research on this topic, you can do so here.