But there are also less known forms of synesthesia. One of our case studies, Megan, experiences music by touch. When she listens to music played on a piano, she feel the sounds literally “poking” her face. Strings vibrate in her chest. Waves from brass instruments pass in front of her, sometimes buzzing around her neck. Drums come up from below. Intensity increases with volume but these sensations are never unpleasant, as Megan feels like she is in the music.
My own synesthesia once saved my life. Since I was a child I have had vivid visual images in response to fearful or uncomfortable thoughts. Intense ones, that is. The fear-induced images take the form of highly wrinkled bluish-greenish paper moving around in an irregular pattern. It’s a whole landscape. Sometimes the images consist in large quantities of quickly presented irregular and wrinkled pieces of bluish-greenish cloth moving around very quickly. Not all of my uncomfortable or fearful thoughts are associated with this sort of phenomenology but the occurrence of this kind of phenomenology is a sure sign of uncomfortable or scary thoughts. The visual phenomenology gives rise to further changes in my body. The images themselves create anxiety and nervousness. When I was a child I used to be deadly scared of the moving wrinkled paper and cloth in my head.
In March 2008 I was hiking in a rainforest in Australia. Snakes were sunbathing everywhere, poisonous brown snakes. Brown snakes are considered the second most venomous land snakes in the world. They have a neurotoxic venom that can cause death to humans within half an hour if the person is left untreated. Even the baby snakes have enough venom to kill an adult human. When hiking in the rainforest there is usually no way of quickly receiving anti-venom. So avoiding snakebites is paramount. Usually it’s easy to avoid the snakes. The hungry and alert snakes slither away when they hear footsteps. The satisfied and lazy, sunbathing snakes don’t feel like moving at all, though occasionally you will need to make them move by throwing little sticks at them, because the trail is so narrow that you can’t step around them.
Back in 2008 when I was hiking in one of the many amazing Australian rainforests my synesthetic bluish-green image had never been unaccompanied by fear. But as I was hiking down a narrow trail, I suddenly couldn't see anything but bluish-green rotation—projected out in front of me. It blocked my visual field. I was blinded by it and stopped. This had never happened before. I didn’t experience any fear, yet my visual field was one big landscape of fear synesthesia. After a few seconds the colors faded enough for me to look around. And there it was: A curled up brown snake right next to me. It was huge in comparison to most of the brown snakes in the rain forest, and it kept hissing at me. I felt the panic in my chest.
“Don’t move,” I told myself.
The snake kept hissing.
I wanted to run. But I knew that running was the wrong thing to do. I was taught this years earlier. If you run, you may scare the snake and it may attack, and if you get bitten and you are running, the venom will kill you faster.
“Don’t move,” I kept telling myself.
Somehow I managed to stand completely still until the snake disappeared. Maybe it was only ten minutes but it felt like hours.
That was when I started loving my fear synesthesia. It saved my life. Or at least, that’s what I like to tell people.
One mark of color synesthesia is that the synesthetic colors are seen either as projected out onto the world (“projector synesthesia”) or in the mind's eye (“associator synesthesia”). Another mark is that it exhibits test-retest reliability: Colors identified by the subject as representative of her synesthetic experiences relative to a given stimulus in the initial testing phase are nearly identical to colors identified by the subject as representative of her synesthetic experiences relative to the same stimulus in a retesting phase at a later time.
Because of the automatic nature of synesthesia and its test-retest reliability, color synesthesia is not to be confused with memory associations or stereotypical colors of objects. For example, there is no evidence that color synesthetes simply remember the colors of entities or images they were exposed to earlier in their lives or associate stimuli with their stereotypical colors.
Synesthetic color experience is unique for each synesthete. For example, the letter A may trigger the color red in one grapheme-color synesthete but trigger the color blue in another. In fact, each grapheme has been found to trigger each of the 11 Berlin and Kay colors in different synesthetes (red, pink, orange, yellow, green, blue, purple, blown, black, white, gray). Despite the uniqueness of synesthetic color experience, synethetic colors sometimes fall into certain clusters. For example, grapheme-color synesthetes tend to associate A with red, E with yellow or white, I with black or white and O with white.
So what is this odd phenomenon called synesthesia, really?
But what exactly is this odd phenomenon called synesthesia? Is it a way of seeing the world? Or is it like visual images in your mind? One way to look at it is as a kind of misperception of the world. What we call an illusion is a misperception of a trait of an actual object. A hallucination, on the other hand, is the perception of an object that isn’t there. A grapheme-color synesthete who sees the number three as green is in some sense misperceiving a trait of the number three. Numbers are not really colored. So her experience is in some sense illusory. A synesthete who experiences non-existing landscapes in response to fear, on the other hand, is seeing something that doesn’t exist. So her experience is hallucinatory.
Many synesthetes know that the world isn’t the way that they see it. This phenomenon is also called a “known illusion” or a “known hallucination.” In the Müller-Lyer Illusion you have a perceptual appearance of two line fragments being of unequal length despite the fact that you know they have the same length.
This is called a “known illusion” because you know that what you experience isn’t so. Some synesthetes, particularly children, do not know that the world is not as they experience it. This is evident in an interview we conducted with one of our research participants RS (at age 5):
RS: Sometimes I see it. Sometimes it’s out in front of me.
E: Is it like seeing something or thinking that something is the case?
RS: It’s like seeing something, and my brain is telling me.
E: Is it exactly like seeing something?
RS: Both, you know...
E: Can everyone see the same colors as you can when they think about numbers?
RS: Not everyone, because not everyone thinks very well about numbers.
E: Are the colors in your head?
RS: Yes, and I am seeing them too.
E: Are numbers printed in colors?
RS: No, they are printed in black but that’s because they don’t know what colors they are.
Synesthete Patricia Lynne Duffy, too, tells us that she thought everyone experienced graphemes in the same colors as she did. The turning point came when she was sixteen:
I was sixteen when I found out. The year was 1968. My father and I were in the kitchen, he in his usual talk-spot by the pantry door, my sixteen year-old self in a chair by the window. The two of us were reminiscing about the time I was a little girl, learning to write the letters of the alphabet. We remembered that, under his guidance, I'd learned to write all of the letters very quickly except for the letter 'R'.
“Until one day,” I said to my father, “I realized that to make an 'R' all I had to do was first write a 'P' and then draw a line down from its loop. And I was so surprised that I could turn a yellow letter into an orange letter just by adding a line."
“Yellow letter? Orange Letter?” my father said. "What do you mean?"
“Well, you know,” I said. “‘P’ is a yellow letter, but 'R' is an orange letter. You know—the colors of the letters.”
“The colors of the letters?” my father said.
It had never come up in any conversation before. I had never thought to mention it to anyone. For as long as I could remember, each letter of the alphabet had a different color. Each word had a different color too (generally, the same color as the first letter) and so did each number. The colors of letters, words and numbers were as intrinsic a part of them as their shapes, and like the shapes, the colors never changed. They appeared automatically whenever I saw or thought about letters or words, and I couldn't alter them.
I had taken it for granted that the whole world shared these perceptions with me, so my father's perplexed reaction was totally unexpected. From my point of view, I felt as if I'd made a statement as ordinary as “apples are red” and “leaves are green” and had elicited a thoroughly bewildered response. I didn't know then that seeing such things as yellow P's and orange R's, or green B's, purple 5's, brown Mondays and turquoise Thursdays was unique to the one in two thousand persons like myself who were hosts to a quirky neurological phenomenon called synesthesia (Excerpt from Blue Cats and Chartreuse Kittens).
So synesthetes are not always aware that their experiences are different from other people’s. However, despite holding idiosyncratic beliefs about the world, they are not typically delusional, as they do not normally hold onto these beliefs once they realize that they see the world differently from most people. Synesthetic experience should also be kept apart from schizophrenic experience. A basic mark of schizophrenic experience is that the deluded attributes the thought or experience to a foreign agent. Unlike schizophrenics, however, synesthetes don’t have the feeling that a foreign agent has planted the vision or sound in their head. Their synesthesia is merely a part of how they experience the world.
Synesthesia may cast light on other seemingly unexplained sensory phenomena, many of which are believed to be spiritual in origin. For example, one mystery is how some so-called healers and psychics are able to reliably describe people’s auras. An aura is a supposed energy field of luminous radiation surrounding a person as a halo, imperceptible to most people.
At the 2011 Toward a Science of Conscious Conference in Stockholm, I stumbled upon Naama Kostiner, a self-proclaimed psychic and synesthete. Naama was able to provide descriptions of dozens of people’s auras and then repeat every word a few days later. Her claim that she actually did see people’s auras was very convincing. The reliability of her descriptions called for an explanation.
Recently Emilio Gómez Milán and his colleagues in Spain found that the ability to see people’s auras probably is a form of synesthesia, which they call “emotional synesthesia.” To carry out the study, the researchers interviewed synesthetes, including a “healer” from Granada, Esteban Sánchez Casas, known as “El Santón de Baza.” The local people believe El Santón has paranormal powers but the researchers found out that what he has is a form of synesthesia. El Santón has face-color synesthesia and touch-mirror synesthesia. Face-color synesthesia arises when there is neural connection between face processing and color. For these synesthetes, different faces are associated with unique colors. Touch-mirror synesthesia is a phenomenon where a person feels the same as a person she can see being touched. For example, if a touch-mirror synesthete sees you being touched on the right shoulder, she too will feel a sensation of being touched on the shoulder.
The latter kind of synesthesia may be interestingly related to the rubber hand and out-of-body illusions. The rubber hand illusion is easy to generate. Put one of your arms behind a screen or box on the table, so you can’t see it. Put a fairly realistic rubber hand on the table in a position that will make it look like it’s your hand. Then ask a friend to stroke both your real hand and the rubber hand in the same way while you look at it. In most cases people have the sensation that the rubber hand actually belongs to them. If the friend suddenly hits the rubber hand very hard, they jump.
Swedish neuroscientist Henrik Ehrsson, the first to demonstrate the rubber hand illusion, managed to induce out-of-body experiences in normal individuals. Thirty-two participants were wearing a head-mounted display connected to video cameras in such a way that the images from the left and right video cameras of a life-sized mannequin were presented to the participants’ left and right eyes. The two cameras were positioned in such a way that the images from each of them corresponded to the mannequin’s eyes. The researchers would then stroke each participant's abdomen and that of the mannequin identically. After doing this for two minutes the participants perceived the mannequin’s body as their own. The researchers then pretended to cut the mannequin’s body with a knife. The participants showed much greater increase in anxiety compared to controls whose stomachs were not stroked in the same manner as the mannequin’s.
The rubber-hand and out-of-body illusions illustrate how sight, touch and proprioception, or bodily sensations, combine to create a feeling of body ownership. In touch-mirror synesthesia, seeing someone else being touched apparently leads to a feeling of being touched in the same place. The illusion apparently is active in these synesthetes without conditioning by rubber hands and mannequins.
Most cases of synesthesia run in families. Synesthete and painter Carol Steen describes the time when she learned her father had synesthesia:
I came back from college on a semester break, and was sitting with my family around the dinner table, and—I don't know why I said it—but I said, “The number five is yellow.” There was a pause, and my father said, “No, it's yellow-ochre.” And my mother and my brother looked at us like, “This is a new game, would you share the rules with us?”
And I was dumbfounded. So I thought, “Well.” At that time in my life I was having trouble deciding whether the number two was green and the number six blue, or just the other way around. And I said to my father, “Is the number two green?” and he said, “Yes, definitely. It’s green.” And then he took a long look at my mother and my brother and became very quiet.
Thirty years after that, he came to my loft in Manhattan and he said, “You know, the number four is red, and the number zero is white.” And he said, “The number nine is green." I said, "Well, I agree with you about the four and the zero, but nine is definitely not green!"
Though most cases of synesthesia appear to be developmental, acquired cases have also been reported following traumatic brain injury, damage to the brain’s white matter, strokes, brain tumors, posttraumatic blindness and diseases of the optic nerve and/or chiasm.
The Neural Mechanism of Color Synesthesia
The precise neural mechanism underlying synesthesia is unknown. Most hypotheses concern the most studied type of synesthesia: color-grapheme synesthesia. One theory is that color-grapheme synesthesia arises due to cross-activation between color areas in the visual cortex and the adjacent visual word form area.
A second is that color-grapheme synesthesia may be due to disinhibited feedback from an area of the brain that binds information from different senses. The fact that synesthetic experiences can arise when subjects are under the influence of psychedelic, such as LSD or psilocybin, provides further evidence for the disinhibited feedback hypothesis. The synesthetic effect of psychedelic substances may be due to an inhibition of feedback from areas of information binding. It is unknown, however, whether drug-induced synesthesia and congenital synesthesia have the same underlying mechanism.
It is plausible that different forms of color synesthesia proceed via different mechanisms. Cases of color synesthesia have been reported in which the visual cortex is not involved in generating synesthetic colors. None of the three aforementioned hypotheses, despite their plausibility in run-of-the mill cases, can explain more unusual cases of color synesthesia.
Cognitive Advantages of Synesthesia
If pop-out effects require attention to the synesthetic graphemes, color-grapheme synesthesia is unlikely to give subjects much of a cognitive advantage in visual search tests. However, there may nonetheless be cognitive advantages associated with color synesthesia. For example, some case studies suggest that grapheme-color synesthetes may have greater recall ability for digits and written names when compared to non-synesthetes.
In rare cases color synesthesia has been associated with extreme mathematical skills. Daniel Tammet, for example, sees numbers as three-dimensiona,l colored forms. His synesthesia gives him the ability to multiply high digits very rapidly. He reports that the product of multiplying two numbers is the number that corresponds to the shape that fits between the shapes corresponding to the multiplied numbers. Tammet’s color synesthesia also gives rise to extreme mnemonic skills. Tammet currently holds the European record in reciting the decimal points of the number pi. An fMRI study comparing Tammet to controls while attempting to locate patterns in number sequences indicated that Tammet’s synesthetic color experiences occur as a result of information processing in non-visual brain regions, including temporal, parietal and frontal areas.
In previous work we have described the case of Jason Padgett, who has exceptional abilities to draw complex geometrical images by hand and a form of acquired synesthesia for mathematical formulas and moving objects, which he perceives as colored, complex geometrical figures.
Jason’s synesthesia began in the wake of a brutal assault that led to unspecified brain injury. A fMRI study contrasting activity resulting from exposure to image-inducing formulas and non-inducing formulas indicated that Jason’s colored synesthetic images arise as a result of activation in areas in the temporal, parietal and frontal cortices in the left hemisphere. The image-inducing formulas as contrasted with the non-inducing formulas induced no activation in the visual cortex or the right hemisphere.
These two unusual case studies suggest that at least some forms of color synesthesia can give rise to cognitive advantages in the area of mathematics. As the visual cortex does not appear to be directly involved in generating the synesthetic images in either subject, the two cases also suggest that at least some forms of color synesthesia are best characterized as forms of high-level perception that proceeds via a non-standard mechanism.