Issue link: https://digital.copcomm.com/i/147928
RENDERING different keyframe pairs in a sequence of 400 or 500 frames. All the frames not painted will be transformed by the algorithm." This means that artists needn't draw on every frame to produce an illustrative style; they simply paint a few keyframes. The technique takes care of the in-betweens. And, the temporal coherence. Under the Hood "One of the underlying existing technologies we use is texture synthesis," Kass explains. "The idea is that here's a texture, make more of it. More specifically, we use guided texture synthesis: Make more of it and let me tell you where it should be and roughly how it should look. That's one of the foundation technologies. We use it to maintain the style of the in-betweens." "Then, since we have an underlying 3D animation," Kass continues, "we render a velocity field for every frame that tells us where every pixel came from and where it will go. We use that information to modify traditional texture synthesis algorithms. The textures we synthesize have the smooth temporal textures we need." There's more. "We use the painted keyframes as constraints on the texture synthesis," Kass says. "We have certain pixels on some images the artist painted that we lock down. The texture synthesis can't change those. Essentially what the technique does is try zillions of ways to chop up the images and rearrange them to come as close as possible toward meeting goals." There are a series of goals. One goal is to make points on different frames corresponding to the same point on an object look as much like each other as possible. Similarly, another goal involves making regions of final frames look like parts of the 54 ■ CGW Ju l y / Au g u s t 2 0 1 3 ■ AT LEFT, A FRAME of Charles Muntz from Disney/Pixar's Up with simple CG shading. At right, the same frame rendered using a Rembrandt etching as the example style. corresponding examples from keyframes. The technique combines the series of such goals as these into a final score with an aim toward achieving the highest score possible. "The underlying 3D animation carries it along," Fleischer says. "It might start with a character that doesn't have much texture or lighting, and from that derive position and motion. The velocity vectors determine how to move the pixels from frame to frame." The solution isn't calculated one frame at a time, however. The technique tries to solve the whole problem – all the frames in the sequence. "The technique makes a series of sweeps over the entire animation sequence, going from beginning to end, back and forth, until it gets a solution for the whole sequence," Kass says. In Practice With the current algorithms, a 30-second sequence might require two hours to calculate and render, although that time would depend in part on how the artists organize the project. "The artists can break the sequence into layers and process the layers independently," Forrester says. "The key reason for that is because we want to give them freedom." For example, using layers, an artist might add keyframes only for the eyes or create outlines that extend beyond the edges of the underlying 3D animated sequence to create a looser style. "To do this well, you need to run rendering tests to get an idea of what the style looks like," Kass says. "We started with the animation and then painted the keyframes because if you change the animation, you need to change the painting." The idea for the skater in the animated clip created to demonstrate the technique came from Director Teddy Newton, who had combined 3D and hand-drawn animation techniques for his Oscar-nominated short film "Day & Night." In the clip, the lighting on the skater changes as he glides through a spotlight. "In the underlying CG animation, the skater gets brighter and yellower as he moves through the light," Fleischer says. "When we do the synthesis, that underlying rendering affects the transitions, so he gets brighter." With this technique, artists can paint the illumination on keyframes, choose to have underlying lighting contribute to the final style, or use some combination of both. "You might want to have different styles for different elements or for part of a scene," Kass says. "You can have one style transformer, or rather than one abstract style transformer, you can nail styles to particular elements. The system can work with a single keyframe, or you can add more and more keyframes." Now that the technique is ready for its close-up, the team hopes that someone at Pixar will adopt a painterly style for their next short film. That's the next test. "That's what we've been hoping for from the beginning," Fleischer says. And then, perhaps, we will see CG features in many new artistic styles, at last. ■ CGW Barbara Robertson is an award-winning writer and a contributing editor for CGW. She can be reached at BarbaraRR@comcast.net.