Issue link: https://digital.copcomm.com/i/147928
■ THE TWO ARTIST-PAINTED keyframes in the top row, at far left and right, provided the style for the automatically generated in-betweens. The bottom row shows the original CG-rendered sequence that guided the interpolation. Video: Go to "Extras" in the July/August 2013 issue box .com moveS Jitter Begone "One of the big challenges with any non-photorealistic rendering (NPR) technique is creating temporal coherence," says Michael Kass, senior scientist at Pixar. "If you take any style and process the frames individually, you get jitter, shake, boils, and pops. You need coherence between frames. That's why hand-drawn animation with texture has excess movement. Each frame is slightly different. You might want a little difference for artistic purposes, but with traditional techniques, you can't avoid a huge amount. For a film the length of a feature, it's difficult to watch. You want something easier on the eyes." Now, Kass and a team of scientists in Pixar's New Looks Project group have replaced that problem with an opportunity. An opportunity for artists and animators to create CG features in entirely new, expressive styles. At SIGGRAPH, the team will show an animation clip in which a 3D character that looks as if an artist had painted it with watercolors, smoothly skates through an atmospheric background. There are no jitters, shakes, boils, or pops. The video accompanies a landmark technical paper titled "Stylizing Animation By Example" by Pierre Bénard, Forrester Cole, Michael Kass, Igor Mordatch, James Hegarty, Martin Sebastian Senn, Kurt Fleischer, Davide Pesare, and Katherine Breeden. "The paper describes two important contributions," Kass says. "One is the temporal coherence, the fluidness of images from frame to frame, and the other is the artistic control that you get by painting keyframes." Pixar Senior Scientist Kurt Fleischer, who served as a technical director for the project, describes its evolution. "Michael [Kass] was working on image filtering when some people in production became interested in the artistic images. That was the genesis. We experimented with different kinds of techniques that looked nice and continuous over time, but we didn't have artist-specific control. We only cracked that in the last couple of years." A SIGGRAPH 2001 paper titled "Image Analogies" by Aaron Hertzmann, Charles Jacobs, Nuria Oliver, Brian Curless, and David Salesin provided one starting point. In that paper, the authors introduced the idea of using two images to define a style transformation. "Their work was with still images, and if you did that transformation on a series of images, you'd get temporal incoherence," Kass says. "So, one of the first things we did was to try to make that approach work in animation, to make it work over time." Making that leap to animation was difficult. "Michael [Kass] shepherded the work over several years with graduate students and postdoctoral students," Fleischer says. "Toward the end, it was Forrester [Cole] and Pierre [Bénard]. There was a nice moment when Pierre and Forrester were here at the same time. I felt like we had critical mass." Artistic Control About two years ago, the technology had moved far enough toward solving the problem of temporal coherence that the team could consider the second goal: building in artistic control. "The project was at an advanced stage when I started working on it two years ago, but the art direction aspect hadn't happened yet," says Cole, then a postdoc student who joined Pixar in July 2011. "We wanted to move it from a research prototype." To provide the art direction essential to move the technology from the lab into production, the team decided to introduce keyframes. "We use select keyframes from a traditional 3D animation that are lit and textured in a simple way," Kass says. "The artists can paint over them in any way and the technique comes up with a series of in-betweens that maintain the style while achieving temporal coherence." In practice, an artist might select keyframes from a simply rendered animated sequence, load them into Adobe's Photoshop, and paint over them to create an output layer. "The simple renderings and the painted images form style transformation pairs," Cole explains. "There can be an arbitrary number. You might have 10 or 20 CG W July / August 2013 ■ 53