Issue link: https://digital.copcomm.com/i/1538125
person's face. It's an incredibly freeing mechanism for adjusting your story in the edit creatively," noted Hall. Performance Transfer lets editors take an actor's facial performance from one shot — whether it's from the same scene or a different one — and apply it to another. For instance, if the most emotionally resonant delivery is in a mid-shot but the preferred framing is a close-up, the editor can blend the two, preserving the best performance in the ideal composition. "So you get the best performance in the frame size that you want, which allows you to structure the scenes exactly how you want and have the best performances," Hall said. Another example of using Performance Transfer is to take an actor's performance (including dialogue) from a deleted scene and transfer it to a shot of the actor being used in the cut. "It doesn't matter about l i g h t i n g c o n d i t i o n s, o r h e a d p o s e, o r anything like that. You're transferring the actor's performance, not the image," added Hall. Performance Editing allows the editor to control the pace of a dialogue performance, thereby controlling the pace of the scene. Hall explained, "In the editing room, we always have instances where the rhythm of the performance wasn't quite right. Or you want to hold for a beat at the end of a line of dialogue but the actor goes to the next line. With DeepEditor, just by editing the audio, you can alter the pace of the scene. The newly-paced dialogue is used to generate a new visual performance." DeepEditor is only changing the area below the eyes, in the mouth region; it's not changing the overall physicality of the actor. "If you have physical emphasis points and physical efforts that correlate to emphasis points of the dialogue, then you don't want to go outside of that. If you have a disconnect between the overall physicali- ty and the placement of the dialogue, then it's difficult to make it sit believably," noted Hall. "I always have a very simplistic way of assessing whether or not a piece of dialogue or new dialogue placement is going to work. I physically put my hand over the mouth on-screen and watch the eyes and the body movement. If I believe that they could have said that, then it will work." With all of DeepEditor's uses — whether it's using an actor's ADR performance, their on-set performance, or an alternative shot of their performance — the key is that the generated vubs are driven by that actor's performance. FINE-TUNING RESULTS Since the goal is to keep as much of t h e o r i g i n a l p e r fo r m a n ce a s p o ss i b l e, editor-created refinements can be applied to the "out of the box" vub, and the Inter- polation timeline is one of the key controls at the editor's disposal. In the Interpolation timeline, when the value is set to 0, the ed- itor is only using the original performance. When the value is set to 100, they're using all of the "selected" performance. The value can be adjusted to any number between 0 and 100. By changing the value at different points in this timeline, the editor can refine the transition between the original perfor- mance and the "selected" performance. Hall said, "It's not a cross-dissolve or anything like that. It's a change in the facial expression. You're adjusting the amount of change, and working with that ethos of only changing what needs to be changed and retaining as much of the actor's original performance as possible." Another feature is the Scale timeline, w h i c h a d j u s t s t h e p e r f o r m a n c e s i z e . Maxing out "scale" would make the per- formance loud and expressive, more like yelling. Reducing "scale" would make the performance more of a whisper. Hall ex- plained, "You might have an audio-driven adjustment. So in In-Vision ADR, you may want to increase the performance size of the dialogue. We try to retain the original performance in our Driving Data as much as possible, but if you're changing a line of dialogue, you're inherently giving that line a slightly adjusted meaning. Otherwise, you wouldn't be changing that line of dialogue! And if you're retaining the original 'perfor- mance size,' then that might not be the right performance for the new line of dialogue. You might want to adjust the performance to become slightly bigger or slightly small- er, depending on the new meaning of the new line of dialogue." The editor can also adjust playback speed. "When an actor is talking fast, some- times it helps to perceive how accurate the lip-sync is by playing the clip at half-speed," noted Hall. DEEPEDITOR FOR ON-SET PRODUCTION While DeepEditor is typically imple- mented in post-production, Mann noted that it can also benefit a production on set. He said, "Performance Transfer, in par- ticular, heavily changes how you direct on set, how you plan, and how you even set up the project. We're only just starting to see the reality of that on a few productions going on at the moment that are using this technology." Mann explained that as a director on set, he shoots the master and then all the coverage separately, angle by angle, trying to get continuity to match the master shot. "You've got this tremendous amount of rep- etition, which is why it takes half a day to a whole day to shoot one scene. No one enjoys the repetition — actors don't, and directors don't. It's part of our old process. What we're seeing now, using DeepEditor on set, is that directors can say, 'Oh, actually, I got the performance in that mid shot, so I only need one take of this angle and it's fine. I can move on.'" He continued, "As a director, DeepEditor is this empowering new tool. It makes the directing job — the on-set aspects — so much easier and faster. We have these new tools that affect all stages of the filmmaking process and that expand filmmakers' capa- bilities while lowering costs, changing the types of projects filmmakers can choose to create, and how they approach production." FLAWLESS A.R.T. (ARTISTIC RIGHTS TREASURY) As mentioned, Flawless designed its AI tools with consent and ownership of rights as a core principle. They developed the Artistic Rights Treasury (A.R.T.) – a rights management system for generative AI — to protect artists and rightsholders and en- sure ethical and fair use of the technology. A.R.T. tracks and links consent, providing 38 C I N E M O N T A G E T E C H SEE PAGE 49