How Film Music Shapes the Storyline

Reflecting on his futuristic 2002 film Minority Report, Steven Spielberg said “one of the most exciting scenes” he had to shoot was this action sequence—in which two characters (John and Agatha) cross a busy shopping mall with armed police in pursuit, and rely on Agatha’s ability to see into the future to successfully evade capture.

As the two figures enter the mall, an instrumental version of Henry Mancini’s ballad ‘Moon River’ can be heard. The music sounds distant, as if playing over loudspeakers inside the shopping mall. Over the faint music, we hear Agatha’s breathless instructions to John as they make their way through the mall, and the heavy footfall of uniformed police searching after them. The gentle ‘Moon River’ serves as a serene musical canvas to the winding tension and disjointed action of this scene.

But what if ‘Moon River’ were not playing inside the shopping mall? What if the same ‘Moon River’ music not ‘inside the scene’—but instead, presented as music that is just accompanying the scene? That is, as a dramatic score—such as the music that typically accompanies a chase sequence?

Would the same piece of music work as a dramatic score? And if so, would it change our impression or reading of the scene?

Film scholars use the term 'diegetic music' to refer to music that supposedly exists within the fictional universe implied by the film. It is music produced inside the world of the characters, that they should be able to hear. In this Minority Report scene, music is diegetic—presented as if playing inside the shopping mall where John and Agatha are on the run.

Another phrase, 'nondiegetic music', refers to music external to the ‘narrative universe’ (or diegesis)—such as the dramatic music often mirroring the mood and punctuating the action of a car chase sequence. Put simply, it is not music existing within the scene as part of the world of the film characters—but simply accompanying the scene.

After a thorough search of existing research, we found that nobody had explored what happens to viewers’ understanding of a film scene if a film music “migrates” from the diegetic world … to the nondiegetic? So my colleagues Matthew Spackman and Elizabeth Wakefield and I set out to investigate!

We used the shopping mall sequence from Spielberg’s Minority Report described above, found at 1:35:33-1:36:57 on the 2002 DVD. We prepared three versions (which we cannot post due to copyright protection):

(1) The Diegetic version was the original unaltered Spielberg version with 'Moon River’ sounding as if it were playing over speakers inside the shopping mall.

(2) The Nondiegetic ‘Moon River’ version was one we created. We purchased exactly the same ‘Moon River’ music as a single, and mixed it louder and clearer to sound like a dramatic score. Speech and sound effects from the original were retained, still crisp and audible.

(3) Nondiegetic ‘chase music’ version. As the gentle ‘Moon River’ music is incongruent with a tense film scene, we created another nondiegetic version with John Williams’ music from the soundtrack of Spielberg’s Empire of the Sun Track 10 (2:03-2:30 and 2:42-3:42). This music was originally scored for a chase sequence at 0:39:49 on the Empire of the Sun 2001 DVD. Again, speech and sound effects were retained.

245 college students participated in our study. They were only told that our aim was “to examine how people understand story lines of film scenes”. They were not told that the focus was on film music.

Each participant watched only one version of the film excerpt, and answered questions—including 10 scales about the scene and film characters. They indicated their responses by drawing an ‘x’ along a line (which we later measured from the left in millimeters to convert to numerical responses).

For example:

(d) I think the relationship between the male and female characters is:

________________________________X_______________

Very friendly Very unfriendly/hostile

When we analyzed the responses of 111 participants who reported that they had not seen Minority Report before this study, we found that their impressions differed dramatically—depending on which of the three versions they had seen:

Click to enlarge

(Enlarge to read the 10 scales we used)

Surprisingly, the greatest differences were found for the diegetic versus nondiegetic version of the same piece of music: (Compare solid red line vs. dashed red line).

Specifically: participants who watched the original diegetic version (music sounding like it was inside the shopping mall)

• perceived the scene as more tense and suspenseful
• perceived the relationship between the male and female as more antagonistic, and more unfriendly and hostile
• believed the two characters had known each other longer
• assumed the female was more fearful and suspicious of the male
• assumed the male was more fearful and suspicious of the female
• believed the male character wanted to harm the female
• perceived the female to be less romantically interested in the male
• and perceived the male to be less romantically interested in the female

... than those who had watched the same scene with the same music presented as a dramatic score (nondiegetic).

The diegetic music may seem like music that “just happens to be playing” inside the mall. ‘Moon River’ as dramatic score, on the other hand, may have been read as the commentary of the film-maker. Therefore when paired with this ambiguous scene, the (non-diegetic) romantic ballad may have softened the edges of a suspenseful scene.

The fast-paced and dissonant nondiegetic ‘chase music’ and melodious, flowing nondiegetic ‘Moon River’ also led to somewhat different impressions (blue line vs. dashed red line). For instance, those who watched the scene with the 'chase music' believed the woman wanted to harm the man, while those who watched it with 'Moon River' (played nondiegetically) thought she was out to help him. But the most striking contrasts were a function of the diegetic/nondiegetic manipulation of the same piece of music (solid red line vs dashed red line).

To the best of our knowledge, our 2008 paper was the first published study on this question. (The results reported here are only a small part of the more extensive study). As is true of an experiment, there are limitations to this study and further research should be conducted using other film excerpts.

It is not our intention to draw any specific conclusions about the definitive or predictable effects of diegetic versus nondiegetic music. While gentle nondiegetic music may have softened this scene, it might have added a creepy edge or ratcheted up the tension for another.

Music interacts in dynamic and probably irreproducible ways within the complex ecology of any film scene.The specific effects of “migrating” a piece of music from diegetic to nondiegetic depend on the unique interplay of music and moving images.

However, what our study does suggest is that diegetic versus nondiegetic presentation of the same piece of music can lead to dramatically different perceptions of the overall tension of a scene, the attitudes and motives and relationships of characters, and other elements of a scene that are fundamental to our understanding of the unfolding story.

In effect, each of the three soundtracks created its own narrative—coloring the perceptions of the audience with respect to some of the most important aspects of scene and storyline.

© 2014 Siu-Lan Tan Ph.D

co-author of Psychology of Music: From Sound to Significance (Psychology Press 2010, 2013) and co-editor of The Psychology of Music in Multimedia (Oxford University Press 2013).

Related posts by Siu-Lan Tan

Why faces are so compelling in film:

http://www.psychologytoday.com/blog/what-shapes-film/201311/3-rea...

Film ‘Gravity’ from the viewpoint of a child psychologist:

http://www.psychologytoday.com/blog/what-shapes-film/201310/gravity-developmental-themes-in-space

Note

A different version of this post appears on World Science Festival's 'Art of the Score: Mind, Music, and Moving Image' site at How Film Music Shapes Narrative and the Oxford University Press Blog . The post was rewritten for Psychology Today. Discussion of the specific findings has also been expanded here.

REFERENCES

Tan, S. L., & Spackman, M. P., & Wakefield, E. M. (2008)*. Effects of diegetic and non-diegetic presentation of film music on viewers’ interpretation of film narrative. Conference Proceedings for the 2008 International Conference of Music Perception and Cognition, Hokkaido University, Japan. Australia: Causal Productions. [A full report of the study, worth further analyses, is forthcoming in 2014/2015]

* See also: : Fujiyama, S., Ema, K., & Iwamiya, S. (2012). Effect of the technique of conflict between music and moving picture using Akira Kurosawa’s movies, Proceedings of the Spring Meeting of Japanese Society of Music Perception and Cognition, 85-70. [Published in Japanese]

Schickel, R. (Writer and Director). (2007). Spielberg on Spielberg. (Television broadcast). In T. Brown. New York: Turner Classic Movies.

RECOMMENDED READING

Kassabian, A. (2001). Hearing film: Tracking identifications in contemporary Hollywood film music. New York: Routledge. [See chapter 2]

Myer, C. (2011). Critical cinema. Columbia University Press. [See chapter 1]

Neumeyer, D. (2009). Diegetic/nondiegetic: A theoretical model. Music and the Moving Image, 2. Retrieved Sept 20, 2013, http://mmi.press.illinois.edu/2.1/neumeyer.html.

Smith, J. (2009). Bridging the Gap: Reconsidering the Border between diegetic and nondiegetic Music. Music and the Moving Image, 2. Retrieved Sept 20, 2013, from http://mmi.press.illinois.edu/2.1/smith.html.

Stilwell, R. (2007). The fantastical gap between diegetic and nondiegetic. In D. Goldmark, L. Kramer & R. Leppert (Eds.), Beyond the soundtrack: Representing music in cinema (pp. 187-202). Berkeley, CA: University of California Press.