Verified by 
Dr. Barry Sandrew
In my recent post "Of Spider-Man Movies and Other 3D Thrillers," I shared a question-and-answer exchange with Dr. Barry Sandrew, founder and CTO/CCO of Legend3D, a leading innovator in 2D-to-3D conversion technology. Dr. Sandrew has an extensive background in neuroscience and a roster of 3D conversions for blockbuster hits including Transformers: Dark of the Moon, Hugo, Shrek and the upcoming theatrical re-release of Top Gun. With 14 VFX patents and 23 years of feature film and TV experience, he has a wealth of expertise to further explain how the brain (and audiences) absorb 3D content and why 3D can enhance the movie-going experience.
Following that original correspondence, I asked Dr. Sandrew a broader, more philosophical question--not how we see in 3D but how we perceive it. Dr. Sandrew replied in his always thoughtful way, and offered the following insights.
Q: How does the brain perceive a 3D image?
We know a great deal about how the brain processes 3D information, but how that information reaches the level of conscious perception is more in the realm of neurophilosophy; an academic discipline and that’s a bit too esoteric for my empirical brain. We often forget that vision like our other senses has evolved purely for survival. Therefore, I believe that in order to understand how our brain perceives information from the outside world we need to place it within that context. In fact, I contend that it’s the type of 3D information that we respond to below the level of conscious awareness that is most telling about our visual system and how it perceives visual information. In other words, our subconscious awareness of 3D is a measure of the true nature of stereovision and the purpose for it.
Q: But isn’t subconscious awareness an oxymoron?
Not necessarily. Not if you define subconscious awareness as autonomic or complex motor responses to visual stimuli that do not rise to the level of conscious perception.
There are some survival situations where our responses to threatening visual stimuli must be quicker than what can be processed consciously. An example is moving your head away from an unexpected fast baseball traveling at 90 miles an hour at your head. Our reactions to that baseball must be quicker than it would have taken to reach a deliberate movement. Instead there has to be some feature of the baseball that activated survival motor flight or fight responses. The most powerful feature of the baseball has really nothing to do with the ball. Instead it has to with “disparity,” the two horizontally displaced visual perspectives that we see of the world due to the separation of our eyes and the distance between them.
If you remember from biology class, our eyes send visual information along bundles of nerve fibers or pathways to our primary visual cortex. It’s now known that this visual information subsequently travels from the primary visual cortex to higher portions of the brain where there are neurons that are specifically tuned to the degree of disparity falling within the visual fields of our two eyes. However, we have no idea how activation of these areas reaches a level of conscious awareness.
As I discussed in my blog, 3D Movies and the Primal Brain, there is a curious condition in neuropathology, first described by Dr. Larry Weiskrantz called “blindsight” in which patients with severe damage to their primary visual cortex can nonetheless “see” emotionally charged objects. While these patients claim they see nothing, they nonetheless exhibit physiological responses indicative of emotional recognition of the objects.
Q: What explains blindsight?
Some neuroscientists claim that blindsight is simply the result of tiny islands of spared visual cortex that are no longer fully connected as they would be in the intact brain. Others postulate the more intriguing idea that there are visual pathways, separate from those projecting to commonly known parts of the brain that mediate a type of vision falling below the radar of conscious visual perception. Whether blindsight is the result of different visual pathways or severely degraded normal vision, the search for an explanation of how the blind can “see” in the absence of their visual cortex has stimulated a great deal of research into the possible role that other, previously unknown visual pathways play in “subconscious awareness."
It’s known that information from our two eyes traverses the brain along many different routes including pathways destined for more primitive parts of the brain such as the limbic system, which mediates emotional responses evoked by threatening visual stimuli. In fact there are volumes of data in the scientific literature to support the notion that emotionally charged visual information travels through the brain via several different pathways, some of which do not include the primary visual cortex and conscious awareness. One of these pathways—the Parvocellular Pathway—is metaphorically called the high road. It sends data-rich visual information (color, texture, shape, etc.) that we perceive consciously about what something is to higher portions of the brain including the visual cortex. The other pathway—the Magnocellular Pathway—called the low road, sends sparse visual information (edges, location, direction and speed) about where something is in spatial relation to our body to primal neural structures lying deep within the brain. It is thought that this later pathway often operates in the absence of conscious awareness.
I contend that the amount of disparity, the direction of change in disparity and the speed of that change is a key element in low-road activation and transmission.
Q: Can you postulate a mechanism?
We know that one of the first areas of the brain to receive low road information is the superior colliculus, lying deep within the brainstem. This is an area that controls gaze and head turning and can subconsciously trigger rapid head and eye movement, forcing us to converge toward an immediate danger situation. Significantly, the majority of brain cells in the superior colliculus are specifically tuned to react to variations in the disparity coming from our two eyes. In fact neurons have been described in the superior colliculus that respond selectively only to “near” objects and other neurons only to “far” objects. It’s not likely that the disparity information reaching the superior colliculus results in the fusion of the two perspectives from our eyes into a 3D image. Instead, I believe that the data received by disparity-tuned neurons in this brain structure is processed to determine relative distance, direction, and speed in the absence of constructed 3D images.
These disparity sensitive neurons in the superior colliculus subsequently send processed disparity information directly to the amygdala in the limbic system as well as to other primitive brain structures that have been well established as modulating emotion. In extreme survival situations, the activated amygdala is known to trigger behavioral and autonomic responses (visceral responses below the level of consciousness) that evoke flight or fight reactions such as instantly ducking or moving out of harms way in response to threatening visual stimuli.
The above is admittedly a simplistic view of the way the brain handles instantaneous, emotionally packed visual information. In reality the brain is a complex matrix of neuronal activity constantly in flux and shifting in location, intensity, and frequency depending on many things including attention, sensory input and motivational factors. To focus on one structure within the brain and point to how it functions in isolation from the rest of the brain is not particularly valid.
However, in an effort to better understand the brain and how it functions in response to emotionally packed images and 3D visuals, it is reasonable to postulate from the scientific literature that a key to subconscious visual startle reactions is dynamic disparity, which is likely the most sensitive indicator of position in space, speed of movement. and direction of movement. In analyzing reflexive emotional responses it would appear obvious that the number of way-stations that visual information relevant to survival must pass through has to be limited and those structures that are involved initially should have influence that outweighs other ongoing priorities in brain activity.
In conclusion, disparity or the horizontal displacement of images from our two eyes due to their separation is the most sensitive indicator of position in space as well as direction of movement and speed of movement in relation to our bodies. In its simplest form, this may act subconsciously to trigger critical survival mechanisms or in its most complex form, it can assist us in essential survival tasks requiring intricate manual dexterity such as food gathering, the making and use of weapons, and hunting.
