Perception and Perceptual Illusions

Perception occurs when sensory signals are matched to perceptual templates.

Posted May 01, 2013

One time, when I was in high school, my friend switched sodas on me. I thought I was taking a swig of Coca Cola, but he had replaced it with cream soda. I immediately spit it out, thinking it tasted horrible. But I generally like cream soda. So what gives? The short answer is that my “top down” processing conflicted with the “bottom up” sensory input. To understand what I mean by this, let’s take a moment to think about perception. After clarifying, I will help you “see” what I mean with some common perceptual illusions.

Look around you. What could be easier and more automatic than seeing the computer or the desk or the trees blowing in the wind outside? And yet, despite occurring so effortlessly and automatically, our ability to perceive the world is a truly remarkable thing. As computer scientists have discovered, it is enormously complicated to figure out how to build an object detector. And not only are we able to detect and identify objects and events, but we have an actual first person experience of them!

The basic outline of how perception works is this. Through experience, the mind/brain builds perceptual categories of objects. These categories emerge from basic interaction with the object and, in humans, via conceptual knowledge and naming. These perceptual (and, to a lesser extent, conceptual) categories serve as schema or templates, and perception occurs via the process of matching sensory input patterns to perceptual templates. The matching process is what gives us the experience of figure/ground relationships. Cognitive and neuroscientists attempt to explore the rules by which bottom up sensory inputs are matched to top down perceptual templates to give rise to the experience of the object. Here is the basic schematic.

A schematic by Richard Gregory

Perceptual illusions provide a great way to experience the template matching process first hand. (That is the “hypothesis generator” box—when the template is matched, the hypothesis is confirmed and you experience the object---that is the “qualia” in the diagram).

Let’s start with a simple example. Look at the picture below. If you have not seen this picture before, your eyes will scan around the dots. Subconsciously, your mind is bringing templates to match the patterns. When the match happens, the object “pops” out at you! (The answer is at the end). Interestingly enough, once you see it you can’t “unsee” it. It just is there.

Another great way to see perception as the intersection of bottom up and top down processing is to take a look at pictures that have dual objects or dual perspectives in them. Because you perceive via template matching, you will experience these illusions as flipping back and forth between different objects or the same object from different angles.  You can’t see them both simultaneously because experience emerges from the matching process. So, the figure and ground keep flipping back and forth.

In the above map outlining the processes, you will see the that there are “rules” next to the “hypothesis generator” box.  These refer to the ways in which basic sensory inputs are categorized and organized. For example, when looking at black and white lines, some of our vision receptors fire at vertical lines and others at horizontal lines. The rules form the patterns that organize the basic sensory input. For example, some neurons fire in response to contrast, which in turn can get processed as motion if patterned in a particular ways, such as in this example. 

 Some of the rules for perceiving size involve context and contrast, as illustrated here.  

 Finally, conceptual knowledge is key. If you play a lot of chess, this depiction will immediately jump out at you as “checkmate”, whereas if you do not play the game, it will be a confusing arrangement of vaguely familiar pieces. (If you are somewhat familiar with the game, you might question if this is in fact checkmate—it is). Likewise, you are able to see these words without effort, but if this had been written in Chinese the experience likely would be of odd shapes and symbols but no meaning. 

The bottom line is that as you go about your life, you are constantly (and unconsciously) organizing sensory inputs via rules and then matching them to your perceptual knowledge, which in turn is then translated, analyzed and manipulated via conceptual knowledge, at least when you are surprised or in need of deeper understanding what you are seeing. It is one of those everyday things that we should occassionally remind ourselves how awesome it really is.

Oh, I almost forgot--those black and white shapes? They are a dalmation heading toward a tree.