Of Spider-Man Movies and Other 3D Thrillers
How the brain processes images in three dimensions
Posted Jul 24, 2012
To address that question, I’ve recently had the privilege of corresponding with Barry Sandrew, founder and CTO/CCO of Legend3D, a leading innovator in 2D-to-3D conversion technology. Dr. Sandrew has an extensive background in neuroscience and a roster of 3D conversions for blockbuster hits including Transformers: Dark of the Moon, Hugo, Shrek and the upcoming theatrical re-release of Top Gun. With 14 VFX patents and 23 years of feature film and TV experience, he has a wealth of expertise to further explain how the brain (and audiences) absorb 3D content and why 3D can enhance the movie-going experience.
In response to my general queries, Dr. Sandrew provided a set of questions and answers that were so enlightening, I’d like to share them here, pretty much in their entirety:
Q: How does the brain process a 3D image?
A. As binocular beings, we have two eyes each containing a retina made up of a sheet of specialized cells that are activated by light of various wavelengths. The lens in each of our eyes focuses reflected light from images in the outside world onto its corresponding retina. Because our eyes are approximately 63mm or 2.5 inches apart, the light reflected off an object in front of us hits each retina with a slight horizontal offset. We call this slight horizontal difference between our left and right eyes “disparity,” and it’s disparity that makes things appear 3D to us.
When you open both eyes, the separate images from your left and right eye reach your visual cortex, where they are directed to highly specialized neurons that are tuned to critical parameters of disparity. As a consequence, the images from your left and right eye are fused into one uniquely different image that exhibits the appearance of depth and volume. From there, neuronal processing is spread in a hierarchical manner to associative and other parts of the brain.
Because disparity gets bigger the closer something comes toward us, disparity is also an accurate indicator of both speed and direction. An object that is close to our face has a greater disparity and consequently has more three-dimensional volume than an object several feet away. If an object is far enough away (between 100 and 200 yards) it will appear to have no three-dimensional volume at all.
Q: What does it mean to “see in 3D”?
A: Most of us take 3D vision for granted. We don’t give it a second thought. We receive three-dimensional information about the world around us using double-eye or binocular cues, but we don’t often appreciate the large amount of three-dimensional information that is derived from only two dimensions. When the image of one thing overlaps another, we know that the overlapping object is closer to us. Shadows can give the impression of contour and depth, even from a 2D image. Texture, brightness, and color saturation are all affected by distance in 2D, and consequently they can offer additional three-dimensional cues.
One of the most important two dimensional cues is parallax. The best way to describe parallax is to use the old car window example. When you are in a car looking out the window, trees in the foreground are whizzing by while forests in the distance are moving comparatively quite slowly. This gives us cues about the relative position in space of the trees in the foreground and those in the background.
The actual separation of items in space and their volume can only occur when we have two functioning eyes that are sufficiently aligned to register images correctly on our retinas. The distance between our eyes has likely evolved over time and is appropriate for our particular species. If our eyes were closer together, things would appear bigger and the three-dimensional volume of space would be reduced. If our eyes were farther apart things would look smaller and the three-dimensional volume of space would be much greater. Our binocular vision allows us to do close manual work like making weapons and harvesting food, while also providing us with knowledge of potential dangers around us to better detect predators. That said, the survival value of binocular vision is extremely significant to the longevity of our species.
Q: Why do we have to wear glasses when watching a 3D movie?
A: In a 3D movie two sets of images are projected on the movie screen, a left image and a right image. In order to have the audience experience a movie in 3D, the left image must be directed to the left eye of each member of the audience and only the left eye, and the right image must be directed to the right eye of each member of the audience and only the right eye. There are different ways of accomplishing this and they all involve glasses.
For More Information:
See Part 3 of Dr. Sandrew's interview at 3D Vision and the Brain.
Visit Dr. Sandrew’s blog, Reflections on the Growth of 3D: “People Who Hate 3D Movies Should Get Their Eyes Examined” and “How Hugo Gave One Neuroscientist the Gift of Stereo Vision”