Psychocinematics: Cognition at the Movies
Watching You Watch Movies
Posted Aug 20, 2013
In the opening scene of The Graduate, Benjamin Braddock (Dustin Hoffman) exits a plane and enters an airport terminal. When he steps onto a moving walkway, the camera begins to track alongside at the same pace, which keeps Benjamin's position fixed and isolated to the far right of an otherwise empty screen. The opening credits begin to fill the space as we listen to The Sounds of Silence, Simon and Garfunkle’s anthem to social alienation. Over the soundtrack, a voice from an airport loudspeaker requests: “Please hold onto the handrail and stand to the right. If you wish to pass, please do so on the left.” This beginning anticipates the entire movie which tells a story of a newly minted college graduate entering adulthood without any sense of purpose or direction. Benjamin is moving though he doesn’t appear to be going anywhere.
Filmmakers have developed techniques, largely through trial and error, that enhance our sensations, thoughts, and feelings. Through acting, staging, sound, camera movements, and editing, movies fully engage us. How do filmmakers direct our attention and keep us riveted to the screen? How do editing techniques link events often in a virtually seamless manner? How do movies drive our emotions, instilling suspense, laughter, horror, sadness and surprise along the way? Philosophers, film theorists, psychologists, and recently brain scientists have considered our movie experience, as of course have filmmakers themselves. Why are movies so compelling?
I have recently advocated a psychological science of our movie experience or what I've coined psychocinematics. What sets this venture apart from other ways of understanding movies is its reliance on empirical research, which through experimental analysis we try to discover how moving pictures drive our mental processes. Although a smattering of empirical studies have been conducted in past decades, there has recently been a growing interest in using motion pictures as stimuli. Also with the advent of brain imaging techniques, particularly functional magnetic resonance imaging (fMRI), we can now link mental processes with ongoing brain activity. Indeed, it is now possible to have individuals watch a movie in a fMRI scanner and record the brain regions that are active during the experience. In this way, psychocinematics can connect minds, brains, and experience as we watch movies.
Even if one is not particularly interested in movies a compelling argument can be made for using videos as research stimuli as they offer a more natural way of simulating the way we experience everyday events, particularly compared to standard stimuli used in most psychological studies, which tend to be restricted to words and pictures. Movies as psychological stimuli offer a dynamic analysis of mental processes as they unfold in time. In recent years, notable applications of movies to study psychological processes have been conducted by innovative scientists such as James Cutting, Jack Gallant, James Gross, Uri Hasson, Dan Levin, Tim Smith, and Jeff Zacks.
In my lab, we have used videos to study dynamic facial expressions (see Marian & Shimamura, Amer J Psych, 2013). Let's say you are at a party and you look up from a conversation and see an unfamiliar face turn and smile at you. Your emotions would be buoyed. Now consider that expression in reverse—you look up and see someone smiling at you, but then that person turns and the expression fades to a blank stare. In our experiment, we showed a variety of short video clips in which expressions moved from neutral to happy, happy to neutral, neutral to angry, and angry to neutral. Interestingly, a neutral face that had just been smiling actually looks rather grumpy. Conversely, when an angry face moves to a neutral expression, the person at the end appears to have a rather pleasant disposition.
These effects have the appearance of a perceptual illusion as the very same "blank" expression actually looks different (grumpy or pleasant) depending on whether it started as a happy or angry expression. In fact, we never interpret facial expressions as static, momentary images. In everyday experiences, expressions move within a rich contextual environment as they track and signal social interchanges. Of course, movies play on a character's facial expressions as the plot thickens. Jack Nicholson's evolution into madness in The Shining comes immediately to mind. These dynamic influences reinforce the idea that much of our emotional involvement is a process that involves an unfolding of events. Psychocinematics has the potential of providing a deeper understanding of mental processes as they move in time—both in the movies and in everyday experiences.