Issue link: https://digital.copcomm.com/i/1521235
M OT I O N P I CTU R E S O U N D E D I TO R S 79 positions of the multiple emitters and listeners simultaneously. This approach optimised the perception of objects as existing within their environments and best-ensured cohesion and congruence, although the technique is more optimised but not exclusive for 3DoF/360° video, not large-scale 6DoF multi-verse. When setting this reverberation up alongside naturalness, precipitation of interaural differences is more often a consideration to maximise externalisation. Source Directivity Using audio emitters that track the corresponding visuals can enhance realism, both for the simulation of physical sound- source behaviour and also for interaction with them from the user's perspective, although we've pointed out that this gives lesser enhancement of binaural localisation compared to reverberation. However, in practice, a virtual object that is automatically attenuated and filtered according to physical parameters such as distance and elevation, when the user is looking away, simply sounds more natural. Attenuation and filtering are applied according to subtended zones—inner angle, outer angle, and interpolated transition in between. This defines gain and frequency-dependent radiation relative to the forward direction of the source. Custom adjustments can be applied to the attenuation curves to mitigate the effect of sudden loudness and intensity changes from user movement. The movement-related gain modulation can often lack real-life smoothness, but fine-tuning ensures that the user does not perceive any sudden changes that might break the immersion, presence and ultimately disengagement from the story or a task at hand. A simple act of being able to both approach a large growling creature from in front or behind—and perceive its audio differently—and differently again with head rotation. Technologies, Tools, Techniques, and Perception Ambisonics is a very popular spatialisation system largely because of the computational ease with which it can be decoded to head-tracked binaural playback. Binaural synthesis is required to stage emitters in 3D space, and optimal rendering is context- dependent and can be performed in real time or in advance. Reverberation is a key component in generating convincing emitter externalisation, and in MR, the auralisation of rooms is important for the sonic collocation of virtual objects within them. Spatial mapping and plane detection are improving and will increasingly contribute to this. The conventional audio creation and processing capabilities are also relevant. To deploy these technologies in an authoring pipeline for XR immersive audio experiences, the various software tools are highly specialised, and range from recording studio and game engines to dedicated 3D audio spatialisers and processors. The delivery hardware—typically an HMD is often idiosyncratic and might offer a unique feature set, but may also require bespoke proprietary software to configure experiences. Different projects would quite possibly require varied tools, although the pipeline is broadly similar even for VR versus MR. Rapid obsolescence is an ongoing issue, yet new tools constantly arise with extended capabilities. There are of course, attendant tools such as microphones that contribute to the ecosystem. The application of all these tools is influenced by the associated visual modes, including film versus CGI and presentation in 3DoF or 6DoF room scale. Conventional audio-asset creation, preparation, and sound- design approaches yield the basic sonic palette, and post-production techniques. For example, Atmos and spot effects are a key backdrop, although these might be spatialised in 3D. Tying specific sounds to emitters is important in dynamic head-tracked conditions to ensure optimal multimodal localisation, but the addition of a parallel audio stream to offer head- locked elements is advantageous. The sound stage can be used to draw the user's attention beyond their current gaze, extend the aural environment beyond the visual boundaries and add functional or emotive emphasis to visual objects. Expanding the audio- visual space with sonic emanation from beyond the boundary of the visuals has sometimes been seen as contentious. For instance, in surround sound for traditional cinema. However, the deployment of 360° audio with 180° video demonstrated experiential advantages by dissolving the boundary of the head-tracked audio field during head rotation. UI-SFX provide valuable interaction feedback and might be a function of either hand controller-driven or hand-tracked engagement, especially when combined with haptic feedback.