Issue link: https://digital.copcomm.com/i/1521235
76 M PS E . O R G side of workflow when delivering audio for a linear 3DoF production. Typically, this may be for 180°/360° immersive films captured on a special camera, CGI-based VR experience or a combination of both. The key difference from full 6DoF experience is that with linear content, you can look around in all directions but cannot interact with the narrative as you would typically do in a game. It's still considered to be a multimodal immersive experience although without a full agency and possibility to interact with the surrounding world. A typical immersive film production may be delivered as a single Ambisonic mix with all components embedded, but this approach may carry some limitations in relation to loudness mastering and the dynamic range of the program. An object-based audio workflow such as Dolby Atmos is a current standard beyond surround for traditional films. The preferred approach currently supported by Meta would be two independent audio playback streams to be muxed/ encoded (multiplexed) with the video stream for the final content distribution. First, a 3D audio playback stream (head-tracked), a dynamically spatialised, for example, an Ambisonic stem, mix typically containing the ambiences, dialogue and individual diegetic audio objects and events. Second, a 2D playback stream, a two-channel stem (head-locked) might carry non-diegetic elements. It can be very distracting if VO and music change position as the user looks around in a 3DoF environment, so this is best to be incorporated into a 2D stem. The head-locked elements can be placed centrally, panned as monaural and stereo sources, hence, perceived to be originating in the head. In this case, the score can still offer stereo width as its mix intended, together with maximising clarity. The 2D playback stream is highly suitable to carry synthetically produced low- frequency and high-dynamic range cinematic SFX. The reasoning for the latter will be discussed further. Sound Design and Content Acquisition The content acquisition process somewhat differs from traditional workflows, the on-camera live action or atmos might be recorded with radio lapels as spot mics, along with an ambisonic mic or other spatial audio capture methods such as Equal Segment Array 3D and others. It's worth noting that despite the 180° visuals, the audio is best to be spatialised to 360° to avoid sonic holes in the entire sound field behind the user. This provides a greater sense of presence and envelopment and mitigates perceived discontinuities at the boundaries of the 180º field during a user's head rotation. Whilst the audience would concentrate on forward-facing visuals, the omnidirectional ambient backdrop provides multimodal continuity and immersion and avoids the risk of encountering the 'exit-door effect' for the audience. It is widely attributed that the resolution of such spatial conflicts in a 2D cinema visual, plus a surround-sound scenario comes from a combination of perceptual cognition and memory. In comparison to that scenario, a dynamic head-tracked sound field with a height component that amplifies both cognitive modes, so careful and sympathetic sound design can act to principally dissolve front/rear transition boundaries without introducing distraction. Implementation and Playback Optimisation With head motion, the HRTF- induced attenuation and filtration enhance the perception of emitter motion and hence, authenticity. However, once audio undergoes any such spatialisation via binaural rendering, it can also lose spectral definition and intensity of dynamic range as a function of head position. Binaural spatialisation can improve utterance detection in a cacophonous audio environment which suggests that a spatialised voice-over narration could be more intelligible even against more dramatic volume increases in the HOA content due to the spatial separation. For instance, VO versus music can mitigate spectral- overlap masking in voices, thus improving clarity. This is related to the timbral-fidelity versus spatial- fidelity conflict which can be problematic in music production and other sounds associated with the strong public perception of other genres of content. However, it is interesting to note in the case of horizontal spatial cues, naive listeners have little awareness of spatial-audio fidelity in the front 60° which is interesting to compare to an audio-visual context, where visual information contributes less to frontal sound localisation estimates. It is worth considering that cognition and information retention aspects in each specific context need to be always balanced against aesthetic concerns. For instance, occasionally, some more abstract SFX can be situated or