Multisensory Music

Dr Helen Mitchell is a musician who studies performers and performance. She’s interested in how people describe, remember and judge sounds, in particular when listening to a music performance. Helen will be joined by flautist Naomi Robinson in a free lunchtime event on Tuesday 14 February at City Recital Hall to discuss the complex interplay between sight and sound and how we listen not just with our ears, but also our eyes.

Dr Mitchell says there is an unconscious bias in listening whereby we are influenced by a range of senses. Her research shows that listening is a holistic experience, where the visual experience in particular contributes greatly to someone’s overall understanding of a musical performance.

“Sound perception is usually considered a purely auditory process, but perceptual research into speech and speaker identification confirms that we become aware of sounds through more than just one of our senses,” she says.

“When you remember the individual quality of a particular sound, for example someone’s voice or recognising the sound of their musical instrument over another, you are not just listening but also interpreting the experience through a whole range of multiple sensory experiences – particularly what you see.”

The combination of sight and sound information also holds for music performance, and audiences use both visual and audio information to identify individual music performers. People who see a silent video clip of a performer can reliably match it to an audio clip of the same performer.

“Listening to music may be perceived to be an aural activity, but we also go to concerts where we see performers on stage and we watch music videos. Our visual experience of the music – the way the musicians look and move, the stage design, the lighting and the other people in the audience – adds another layer of meaning to how we appraise and remember the quality of sound afterwards.”

The idea that music listeners need visual cues to listen is a scientific curiosity, but has profound implications for the music profession, particularly for training the next generation of expert performers and listeners. Dr Mitchell believes that awareness of multi-sensory listening is important for those who are regularly judged on the quality of their performance, be it by audiences or future employers.

“If you know that your audience is going to be just as influenced by what they see as well as what they hear, you can be better prepared for an audition. Conversely, I hope that this kind of research will help create better listeners and adjudicators who can make decisions that affect the future careers of young performers.”

To help musicians learn more about the process of listening, particularly in preparation for auditions, Dr Mitchell runs ‘moot auditions’ where she works at the Sydney Conservatorium of Music.

“In a moot or mock audition, music students role play as auditionee, and also as auditioner, and experience music performance at a blind audition, and at a sighted audition, and evaluate the performances. It puts current empirical music research in real-time, so students experience the inherent challenges of expert music listening and evaluation.

Dr Mitchell believes that we are all expert listeners, but we may not yet know it.

“I hope that after participating in my talk at City Recital Hall and experiencing the flute performance by my colleague Naomi Robinson, a fourth year honours student at the Conservatorium, audience members will think more carefully about how about how they listen and appreciate the innate human ability to integrate the senses.”

All welcome to join Dr Helen Mitchell and flautist Naomi Robinson at City Recital Hall on Tuesday 14 February at 12.30 pm for the first in Inspiring Australia’s free lunchtime event series: This Sounds Like Science.

Register to attend

Find out more about Dr Mitchell’s research