Post Magazine

January 2017

Issue link:

Contents of this Issue


Page 39 of 43

SOUNDTRACKS 38 POST JANUARY 2017 V irtual reality and 360-degree videos have been taking the entertainment industry by storm recently. Startups and established content producers are all experimenting with the format, trying to understand what kind of experience in which the general audience is interested. The technological side of the area is also in constant mutation and every project brings new challenges and unknown workflows to the table. At Pomann Sound, we recently worked in partnership with the Big Picture Company on a music video shot in 360-degrees for which we were asked to mix the accompanying music in Ambisonics. In case you aren't famil- iar, Ambisonics is a sound format that allows for very flexible reproduction of the sound field around a point. Instead of assigning channels to speakers, which locks the direction of the sources into place, the sound in this format can be de- coded into any number of speaker arrays in an unlimited number of configurations. It is also extremely versatile because you can encode any other sound sources, such as stereo or mono recordings, into Ambisonics in order to manipulate it. One of the biggest challenges of cre- ating sound for 360-degree immersive experiences is the fact that the viewer can be looking in any direction at any point in the video. In order to maintain immersiveness, it is necessary that the sound stays anchored to this 360-degree world, rotating around the viewer as they look around. That is where Ambisonics excels — once you design and mix your sound, the player can decode it into bin- aural for headphone playback in realtime, guaranteeing great accuracy to the direc- tion at which the viewer is looking. In order to mix the video's immersive sound, we started from a pre-mixed set of stems of the song that was recorded in the studio. We then placed each of the instruments into the 360-degree space to match their positions in the video and decided on the spread of each instru- ment, similar to how it would be done for a stereo mix. The fun begins when it's time to decide how to creatively widen or pinpoint certain instruments or frequencies in this sound field in order to make the viewer aware of the direc- tionality of the sounds or not. One trick we discovered, for example, is to spread the bass frequencies around the viewer using mirrored sources and convolution reverberation in order to simulate how we experience bass frequencies natural- ly, while keeping high frequency sounds such as hi-hats and the synthesizer limited to the position of the instrument in the video, making their directionality easily noticeable as the viewer looks around. This gives the viewer the chance to focus on each instrument separately without losing the unity of the mix. Another interesting challenge of the video was a moment in which we see the musicians playing from the oppo- site side of a studio's control room glass window. The director wanted the sound of the music to come from the engineer's speakers, which can be seen in one side of the room. The challenge was to balance how realistic the sound should be, which is a hard decision to make because the 360-degree video format naturally demands a high level of realism, which doesn't necessarily serve a music mix the way we are used to after years of stereo music consumption. We ended up recreating room reflections coming from different directions in the room and the muffled sound of the occluded music coming from the glass, but everything very subtly so it wouldn't interfere with the music coming from the engineer speakers, which was kept unaltered in order not to radically take the viewer out of the music. The great thing about editing and mixing sound in Ambisonics is that even though the concepts and tools are very different from traditional film sound, the tools are very accessible, so experimen- tation is easy. The entire spatial mix was done using Reaper, a DAW (digital audio workstation) with very flexible multichan- nel capabilities, and a few different sets of Ambisonics encoders, decoders and manipulators, such as the ambiX suite, which is available online for free. The best part is that since the final video is meant to be heard through headphones for full immersiveness, you don't even have to make use of studio speakers for the spa- tial mix, just a good set of headphones. It really all comes down to experiment- ing and understanding the technology and the wide array of possibilities it brings to the creative process. Now that YouTube openly supports playback of Ambisonics in 360-degree videos, this is an area that is going to keep on growing, with more ideas and challenges being tackled by the industry, and we're very excited to be part of this moment where we can work together to determine a new language for VR sound. THE CHALLENGES OF AMBISONICS SOUND EXPERIMENTING AND UNDERSTANDING A TECHNOLOGY WITH A WIDE ARRAY OF CREATIVE POSSIBILITIES BY CLAUDIO SANTOS MIXER/EDITOR/VR GURU POMANN SOUND NEW YORK CITY POMANNSOUND.COM Pomann is now mixing audio in Ambisonics.

Articles in this issue

Links on this page

Archives of this issue

view archives of Post Magazine - January 2017