Issue link: https://digital.copcomm.com/i/1524736
C I N E M A A U D I O S O C I E T Y. O R G 15 LISTENING EFFORT Looking forward, we were introduced to some new and upcoming software that may help predict how a mix will be received by the listener. Fraunhofer IDMT (Institute for Digital Media Technology) gave a presentation about the physiological and psychological ways the human brain processes dialogue, the research they're doing into this, and the software they are developing to measure it. We heard about what can be termed "listening effort," and this is the approach they have taken to approximate the human experience using software that quantifies and measures exerted cognitive effort in real time, which could be integrated into mixing or QC workflows (the latter of which makes me break out in a cold sweat—imagine getting a QC report that failed the mix for a bad "listening effort" score). The way Fraunhofer measures listening effort is based on the phonetic nature of language. Phonemes exist across all languages, so by breaking it down to that level, they ensure that the measurements are equalized across languages, dialects, accents, etc. Indeed, this was my first question to them—I wondered if the meter would show increased effort for a thick accent. It does not, but for good reason—this being Fig.1 Effort Scaling Categorical Unit (ESCU) Scale another highly personal aspect of the viewing experience that it would be impossible to adjust for humanity as a whole. The meter works on averages, which is really all any quantified measurement tool could hope to do. But it brings me back to an earlier point: Younger viewers don't have the requisite life experience to understand every word that an older, more experienced viewer does, and this is also true across cultures. I grew up in the UK and, culturally, I was exposed to the full range of English accents—Irish, Welsh, Scottish, European, and South Asian English-as-second-language speakers with accents, American, Australian, New Zealander, South African, and so forth. Now, I have the troubling legacy of my imperialist forbears to thank for all this, but the point is that I'm just used to hearing all those variances in accents, and the same cannot be said for people from all primarily English-speaking cultures. Many North Americans have a hard time with some regional English accents, and indeed can't correctly identify an English accent vs. an Australian one in many cases, which is not to judge but to say that what I find perfectly clear and distinct is not what someone else would find as such. I understand every single word spoken on Peaky Blinders, but I know of many Americans who do not have the same experience, and I think it would be unfair to claim that this could be solved with different mix choices. Now, back to the meter. It provides a real-time readout of averaged Listening Effort on a scale of 1 to 13 (well, technically 14, but 14 represents "pure noise," so nothing really reads that value unless you're falling asleep to the TV in the '70s). An oddly specific scale at first glance, it's based on the ESCU (Effort Scaling Categorical Unit) from the paper "Assessing and Modeling Spatial Release From Listening Effort in Listeners With Normal Hearing: Reference Ranges and Effects of Noise Direction and Age" by Melanie Krueger, Michael Schulte, and Thomas Brand—if you're inclined to dig deeper. The concept of "effort" is again dynamic and personal—aside from what each individual finds easy or hard to understand based on their experience. For example, what I might consider to be a reasonable amount of effort to put into engaging with a film or show might be different from others. I don't want to imply that viewers shouldn't be expected to exert any effort at all—there's plenty of content you can watch if you don't want to engage your brain while doing so, and that shouldn't dictate how more "cerebral" content is created and mixed. Indeed, in listening effort tests, a 33% variability is observed based on personal willingness to exert effort. This is significant—I myself don't do much if any "distracted viewing," so my attention is pretty focused on the content, and I wouldn't be too happy adjusting my approach to a mix so that a casual, distracted viewer has an easier time following along. But then that does also depend on the content itself, and since not