Issue link: https://digital.copcomm.com/i/1505509
www.postmagazine.com 11 POST JULY/AUG 2023 ARTIFICIAL INTELLIGENCE rtificial intelligence has the poten- tial to transform every aspect of a post production pipeline. Engineered to maximize efficiency and foster creativ- ity, AI features are now available across workflows for broadcast, virtual produc- tion, video editing, visual effects and beyond. Here, we detail innovative new AI offerings from three major companies. Nvidia At SIGGRAPH 2023, Nvidia released approximately 20 research papers fo- cused on the advancement of generative AI and neural graphics. Developed in collaboration with over a dozen universi- ties across the US, Europe and Israel, the papers explore new applications for text- to-image technology, inverse rendering tools for transforming still images into 3D objects, neural rendering models for gen- erating realtime visual details and more. Visual effects artists and digital artists may especially benefit from Nvidia's research into new AI techniques that will automatically convert 2D images and videos into complete 3D representations. This will greatly accelerate the process of transforming concept art into fully-ren- dered environments populated with 3D characters and objects. The technology also streamlines the process of produc- ing 3D characters capable of performing realistic movements and diverse skills without requiring expensive motion-cap- ture data. When animators generate 3D charac- ters, the process of layering in realistic details can be a very time-consuming and computationally-expensive endeav- or. Hair is a particularly complex detail to generate. For example, humans have an average of 100,000 hairs on their heads, and each one reacts dynamical- ly to motion and to the environment surrounding it. Animators typically use physics formulas to calculate hair movement, and must often simplify it accordingly if resources are limited. Nvidia's new research showcases an AI technique that is able to simulate tens of thousands of high-resolution hairs in realtime using neural physics. The process also teaches a neural network to predict the move- ments of each individual hair. Specifically optimized for modern GPUs rather than CPU-based solvers, this efficient new approach to hair generation significantly reduces simulation times from multiple days to just a few hours. Blackmagic Design Specifically designed to speed up audio and video editing workflows, Blackmagic Design has integrated a range of inno- vative AI tools into DaVinci Resolve 18.5. These new and updated features include virtual lighting tools, intuitive object mask- ing, instant depth map effects, automatic audio clip categorization and more. Resolve's new Relight FX tool allows editors to add virtual light sources into a scene, giving them the flexibility to brighten dark shadows or make creative adjustments to environmental lighting conditions. Offering surface softness, specularity and directional controls, these virtual illumination sources can cast a broad light, a point source or a spotlight on demand. Engineered to recognize and track the movement of thousands of unique ob- jects, Resolve's intuitive new object mask isolates countless elements, including people, animals, food, vehicles and more. These capabilities are useful for many effects and grading applications in a video-editing workflow. Audio engineers are now able to use the DaVinci Neural Engine to automat- ically analyze and classify audio clips based on their content. Designed to ex- pedite the often time-consuming process of reviewing unfamiliar materials, the AI-powered engine sorts audio clips into dialogue, music and effects bins. Each bin includes detailed sub-categories, such as insects, water or explosions. Resolve's AI-based voice isolation tool allows audio editors to remove loud, un- desirable background noise from record- ings. Ideal for interviews and dialogue recordings from noisy environments, the tool efficiently separates voices from background sounds. Adobe Designed to simplify video editing pipelines, Premiere Pro now features a text-based editing system. Harnessing the power of Sensei, Adobe's machine learning and AI framework, the intuitive new system is well suited for the rough- cut phase of a video-editing workflow. Premiere Pro's AI-powered, text-based system automatically ingests, analyzes and transcribes video source material. This allows editors to easily skim the transcripts, search for keywords and add them to their timeline to begin a rough cut. Editors are then able to copy and paste phrases into their sequence transcripts, and the edit in the timeline will automatically change to match. The sequence transcript can also be used as a navigation tool throughout the editing workflow. When the editing process is complete, the text-based editing function provides a ready-made transcript for easy caption generation. AI: NEW APPLICATIONS FOR POST PRODUCTION BY KENDRA RUCZAK MANAGING EDITOR, CGW INNOVATIVE TOOLS FOR VFX, ANIMATION, EDITING, AUDIO & BEYOND A Adobe's speech to text feature Blackmagic Design's Resolve 18.5 Nvidia's reseach will benefit VFX artists.