Many of the films we love manage to put us in someone else's shoes, whether it be the shoes of a social network tycoon or a zombie killer. After all, we don't pay $15 to see on screen what we do all day. Writers and directors get us into the protagonist's head in a variety of ways, including letting us hear his or her inner thoughts.

But a more direct route into the shoes of the protagonist is to make it appear as if the viewer is actually in the body of the protagonist. That sounds simple enough: film from the protagonist's vantage point and have the actors pretend that the camera is the protagonist.

Achieving a first-person perspective is, however, considerably more subtle than this. The question with which one must grapple is this: "What does a human view of the world actually look like?" Just because you are, at every open-eyed moment of your life, seeing such a view doesn't mean the question is trivial. What is immediately before our eyes is often the stuff most easily missed.

First-person video games get one step closer to realism by putting the character's arm and weapon into the player's view, providing the visuo-motor feedback of one's arm that is needed for ably pulling levers or chainsawing demons.

But the view from ourselves includes more of ourselves than just our arms and whatever weapon we're wielding. As the father of "ecological psychology," James J. Gibson, pointed out nearly fifty years ago, we can also see our own nose, cheeks and brow. That's why American football players place black streaks on their cheeks -- their eyes see their cheeks, and are otherwise partially blinded by them on bright days.

Neither video games nor movies have gone so far as putting the protagonist's view of his or her own face into the camera view. And there is a very good reason for not putting the character's face into the first-person viewpoint: it blocks the world that the viewer is trying to see. If you close one eye, you'll see that your nose alone takes up a considerable portion of your view. That's a lot of movie screen to fill with a schnoz! And the same difficulty applies to the arm and wielded weapon, which also block one's view of the actors and objects beyond the protagonist's face and arms. No wonder the protagonist's view of him or herself has been avoided in movies. It blocks too much of the world.

And, yet... When we look out at our world with both eyes open, we do not find a nose occluding our view. Instead, our nose is visible to us, but it is rendered as transparent -- we see our nose, but also see through it to the world beyond. (If you can't see your nose, try wiggling it.)

In fact, as you attend to your transparent nose in your visual field, you'll notice that you see not one nose, but two. The nose on your left is the right eye's view of it, and the nose on your right is the left eye's view of it. Also, if you hold up your hand in front of you and look at something far in the distance, you will see two transparent copies of your hand. The amount of the world your hand occludes with both eyes open is small compared to how much is occluded when only one eye is open.

As I discuss in my recent book, The Vision Revolution (Benbella, 2009), one of the central functional advantages of binocular vision concerns "seeing through" our own body parts. How else to put "gangly parts" (e.g., nose and hands) out in front of our eyes, thereby getting visuo-motor feedback, without occluding the world beyond?

A transparent-double-view of our gangly parts is the result. That is, the answer to the question, "What does a human view of the world actually look like?" is roughly this: two partially transparent copies of our nose and hands, through which we see the world.

We can, then, mimic the true human vantage point in movies and video games. And we can do so without occluding the viewer's view of the world beyond.

But... we can only do so by having both eyes open.

For movies and video games that means using "3D". I put "3D" in scare quotes, because now the intent of having different left- and right-eye images is for accurately putting ourselves into the protagonist's shoes, not for getting 3D pop-out effects at all.

And that's the point that 3D movies are currently missing. What everyone appreciates about 3D movies is the more realistic view of the world it provides. What is not appreciated is that 3D movies are capable of simulating for viewers what it really looks like to view out of the eyes of the story's hero.

The greatest experiential advance 3D movies might provide may not be the 3D depth at all, but, I submit, the body-transport powers that bring us to stories of all kind in the first place.


Mark Changizi is Professor of Human Cognition at 2AI, and the author of The Vision Revolution (Benbella Books) and the upcoming book Harnessed: How Language and Music Mimicked Nature and Transformed Ape to Man (Benbella Books).

You are reading

Nature, Brain, and Culture

Just Say No to Artificial Brains

Artificial brains won't happen.

For Better Science Education, STEM Must Die

Scientists and engineers need different motivations.

Language, and How to Have a Fair Fight with a Chimpanzee

In a chimp-human fight, is language allowed?