Issue link: https://digital.copcomm.com/i/1536119
had assumed that it would be feature animation and VFX that were going to be the earliest adopters. That was so not the case, like not at all. It was games. Games had the need for volume and for quality at volume. They had the need for localization and had the pipeline set up so that you could generate all of that animation as you're going. Animation at the time—the reason why it goes: render, then send to dubbing or localization, is because of that render process at the end of a linear pipeline. You're not rendering all the time in real time or in-engine, so you're not able to benefit from the idea of localizing that animation as you go. The input is from the script. At the time we needed both audio and text to feed the analysis system. Those inputs are analyzed. They're broken down into their smallest composite parts—almost like their molecules, which would be phonemes and visemes. Our R&D has al- ways wanted to go subatomic or submolecular, and that's the direc- tion that we continue to go in terms of making richer the data source and then also more editable or controllable the output animation. That analysis is followed by an alignment phase where those com- posite pieces are then matched up with each other. So, visual to au- dio—visemes into phonemes. Then the format is animation curves. The thing that has always made us competitive and continues to do so—alongside the development of AI-emergent or other ways of capturing and then producing facial animation data like p-cap or performance capture—is the curves that our system outputs. That data—if you were to look at it on the graph editor in Maya, which a lot of animators work directly in—it's math and tangents. The curves are as though they had been key-framed, so they're very clean. The tangents on those curves are correctly constructed. You could just continue animating on them. There's no cleanup—is the advantage. That cleanup component is accompanied by an investment. That's a block of time that has to be done, and that's still the same, even with markerless capture, natural movement. When your source that you're capturing is a human performing, that noise that they talk about—which is all of these points per frame—is just natural motion. Most of the time in an animated scenario, you want a cleanliness to the motion. When you think back to the examples of fully perfor- mance-captured feature 3D films—how odd they looked because they had this very natural quality to the movement which is inher- ently noisy and jittery and of humans, of real life. But in this animated, artificial world that's of someone's imagining, it just didn't jive. That's the definition of uncanny, when something does what it's not sup- posed to do. Now, with facial animation, your traditional pipeline is p-cap or a common pipeline: p-cap, cleanup, polish. With us, it's just: animate, polish. The out-of-the-box is going to be of probably lower quality than say, p-cap, but the time to polish is so much more predictable, and faster. So the time to the same end result is much less. Then it's integrated into Maya and Unreal. A game is interesting because the whole thing has to be animated. A view of wherever a player's viewpoint is—that has to be animated. If you look at the ceiling, the ceiling is animated. The same as if you look at a character—they have to be doing something. That's insane- ly expensive, which is why you get typical NPC [non-player charac- ter] movements. As long as they don't do something disruptively out of the ordinary, then you don't notice. Immersion persists uninter- rupted. Immersion and perfection don't have to be the same. It's just JALI jRig control board in Maya (top le & right, below)