Seeing the World Through Music
Listening to music can give shape to our visual perception of the world.
Posted Jun 15, 2018
In his poetic exploration of human perception, “The Idea of Order at Key West,” Wallace Stevens portrays two people listening to a woman singing on the seashore. Mesmerized by the song, the narrator reflects upon the singer's seeming ability to impose order upon the world around her. When the singing ends and the two companions turn back toward the town, he observes that the world looks different to him than it did before hearing her song:
Why, when the singing ended and we turned
Toward the town, tell why the glassy lights,
The lights in the fishing boats at anchor there,
As night descended, tilting in the air,
Mastered the night and portioned out the sea,
Fixing emblazoned zones and fiery poles,
Arranging, deepening, enchanting night.
Hearing the auditory patterns of the singer’s melody has altered the way the narrator processes the scene. A recent study in Spain provides support for Stevens’ portrayal of human perception, demonstrating that listening to rhythmic sound patterns (i.e. melodies) can, in fact, influence the way we process visual information.
Most cultures of the world use visuospatial terminology to describe the purely auditory phenomenon of musical pitch, referring to individual tones as either “high” or “low” relative to other musical tones. Carlos Romero-Rivas et al. sought to discover whether this cross-modal convention is strictly metaphorical, or if perhaps there is a literal basis to the figurative description. Specifically, they investigated “whether the passive listening of melodies biases visuospatial attention towards high and low spatial positions.”
In the study, 19 participants with no musical training were presented with a series of predictable and unpredictable 11-tone melodies. Simultaneously, they completed a visual task that they were told involved determining the color of a circle that appeared either above or below a point at which they had been instructed to stare. Actually, the task was intended to measure the influence of musical patterns on predictions about the location of a visual stimulus.
The circles appeared on the screen at the conclusion of the 11-note melody in place of a hypothetical 12th note. The auditory streams were either predictive (repeated high and low tone patterns that would logically conclude with either a high or low 12th note, if one was played) or non-predictive, or seemingly random note patterns. When the colored circles appeared on the screen in place of the 12th note, they were either above or below the gaze fixation cross at which the participants were staring. For the predictive auditory patterns, the placement of the dots was either congruent (“high” or “low,” based on the expected pitch of the 12th note) or incongruent (a placement at odds with the expected pitch). For the non-predictive patterns, of course, consideration of congruence was irrelevant.
Since the participants had been instructed to determine only the color of the visual stimuli, with no mention being made of location, they were not consciously aware of any significant relationship between the 11-note streams and the placement of the visual stimulus that appeared immediately after them. Regardless of the apparent irrelevance of the melodies to their assigned task, however, participants’ reaction times to the congruent (relative to the incongruent and unpredicted) visual stimuli were significantly faster in the final trials of the experiment than they were in the earlier trials. In other words, hearing 11-note melodies that generated a predictable high or low 12th note, and then seeing a visual stimulus either above or below their line of gaze (depending on whether the predicted note was high or low) trained them to look toward a position in visual space that corresponded to the pitch of an auditory stimulus.
Participants’ electroencephalographic responses to the visual stimuli supported the conclusions drawn from the behavioral evidence. Measurements of event-related potentials indicated a “surprise” response to incongruent visual stimuli, suggesting that, even without their conscious awareness, participants came to expect the colored dots to appear in the position predicted by the preceding 11-note melody.
This study demonstrates that, as Stevens’ poem suggests, listening to predictive melodies spontaneously modulates visual attention, influencing the way in which we process visuospatial information. Hearing a melody can, in fact, affect the way we view visual stimuli in our environment, even causing us to perceive patterns among those stimuli that we did not notice before. Whether those stimuli happen to be reflected lights from fishing boats on the ocean, colored circles on a computer screen, or perhaps even pigments on a canvas or points on a sheet of graph paper, what we hear in their presence exerts a profound influence on what we see when we look at them.
Romero-Rivas, Carlos, F. Vera-Constán, S. Rodríguez-Cuadrado, L. Puigcerver, I. Fernández-Prieto, and J. Navarra. “Seeing Music: The Perception of Melodic ‘Ups and Downs’ Modulates the Spatial Processing of Visual Stimuli.” Neuropsychologia. 2018 May 10; 117: 67-74.
Stevens, Wallace. "The Idea of Order at Key West." Poetry Foundation. https://www.poetryfoundation.org/poems/43431/the-idea-of-order-at-key-west.