Issue link: https://digital.copcomm.com/i/1456671
39 S P R I N G Q 1 I S S U E T E C H John Flynn. Mark Lanza. story development. S o n a n t i c w i l l s o o n o f fe r s p e e c h - t o - s p e e c h c a p a b i l i t i e s f o r A I v o i c e generation, according to Flynn. "This hy b r i d a p p ro a c h i s t h e b e s t o f b o t h worlds," he said. Flynn sees speech-to-speech as a way for Sonantic users to finesse per- formances of specific lines, as opposed to being a main approach to how their software is used. He said, "I would guess that approximately 5% of a script would be edited using speech-to-speech. A lot of our customers will do batch generation in the early stages of a production, and when moving towards a final cut, they will spend time finessing a small portion of the lines to get the performance they need. There may be one line that just isn't quite right, and it's easier to verbally direct how you want it to sound. The software will mimic the exact intonation it is given." Fo r m o r e c o m p l e x p r o j e c t s , S o - nantic can work with clients to create custom voice clones, as they did with actor Val Kilmer to recreate his voice after the damaging effects of throat c a n ce r. Fo r t h i s, t h e y n e e d ro u g h l y three hours of clean material with the right range of coverage, from phonetics to emotional styles. But in the case of Kilmer, they could only use historical recordings. "We had to strip away the background noise while keeping the quality of his voice intact. After cleaning the audio samples, Sonantic was left with ten-times less data than typically needed. This led to the creation of new algorithms, tech- niques, and models that have since been incorporated back into the Voice Engine, strengthening its ability to handle diffi- cult voice content in the future. Overall, we created 40 different options before finalizing Val's prototype," said Flynn. Creating a custom voice clone can take anywhere from six to twelve weeks. But once it's done, the voice model will appear in the user's Sonantic platform alongside the other voice models. Speaking of the Future S u p e r v i s i n g s o u n d e d i to r M a r k A . Lanza, MPSE, President of Motion Picture Sound Editors, sees the potential of this technology for use in legac y projects, "where it's the seventh, eighth, twelfth iteration of a movie, and they want to bring back an actor who had done one a decade ago and has since passed. With the leaps they're making in visual mediums — being able to visually recreate an actor — they can now sonically recreate the actor as well. The implications are amazing. You can have any actor who's ever lived be in your movie. You can't tell the difference visually more and more, and you're not going to be able to tell the difference sonically," he said. As a sound designer, Nelson is excited to use AI-voice technology as a way to create new languages for films, or create new alien-sounding voices. "This creates a whole new realm where you can take this library of particular performances, load it into this program, and use parameters to manipulate the sound to make a whole different language that doesn't exist. You could swap out all the consonants for dif- ferent consonants. This might be the next level, but I hope that we're still keeping that performance — that serendipity — that ex- ists when you have actual humans involved in the process," said Nelson. Serdiuk sees AI voice generation not as a competition for actors but as a way for them to expand their range. "Our tech is heavily reliant on voice actors because they need to perform the part. Having our technology as part of their business, they can get more jobs because they're not tied to the voice they are born with. They can perform in a different gender or different age just by picking a different voice from the library," he said. A s s p e e c h s y n t h e s i s d e v e l o p s a n d improves, it helps establish clearly defined legal and ethical policies that can help deter the deceptive use of this technology. Respeecher is also exploring ways to water- mark content generated by their software. For documentaries, being forthcoming about using voice clones or AI-generated voices could make it more acceptable for au- diences. Maybe that's done by having a title card up front saying an actor's replicated voice was used in a project — as was the case with Savage's latest project. "In my opinion, bringing awareness about the tools is one of the best mea- sures that we can take in order to defend ourselves against the tools being used nefariously," concluded Serdiuk. Jennifer Walden is a frequent CineMontage contributor who specializes in post-production technology. ■