Issue link: https://digital.copcomm.com/i/196542
Motion Capture shell from the scan, and drills holes to show the makeup artists where to paint the markers. But, markers aren't always used – or necessary. Because the video is high resolution and uncompressed, it's helpful for markerless pattern tracking. "We are marker agnostic, Derry says. "We don't care if the " client does marker, markerless, or surface capture. Most of our clients use a technique similar to what we did on Avatar with paint-based markers. But some are doing a surface scan, and some are doing ingenious pattern-tracking that doesn't require markers. We just provide the video. We don't do the processing. There are other outfits good at the processing side. " Typically, a motion-capture or facial-capture supervisor for the project will decide what works best for their production. "Every company approaches the problem differently, and every company has a different marker pattern, Derry says. "There is no " set-in-stone way to do it. With body capture, the technique is pretty much the same whatever system you use. Facial is still up in the air. If you're going to put an actor in the show, a surface-scan technique makes sense. If you have an actor playing an ogre or another character, you might go through the process of applying markers, tracking, solving the source, and then retargeting. Some groups send the video to a service bureau; others do a lot of work in-house. It's still a little Wild West. " Derry believes that with the help of performance capture, we'll see characters performing with photoreal faces in games in the future. "That isn't as far out as people might think, he " says. "The animation side has come leaps and bounds in the past few years. The big trick has been lighting, not animation. " usc/ict High-resolution faces with video performance capture Studios have turned to USC's Institute for Creative Technologies (ICT) for years to capture static expression scans in the LightStage system, which surrounds an actor's face with light. "For the static scans, we ask an actor to hold an expression for two seconds while we turn on different patterns of light and take 16 photos of the face, explains Paul Debevec, as" sociate director for graphics research at USC ICT and research professor at USC. "We have flat, even illumination, and then gradients with the bright light at the top, at the bottom, moving from left to right, and right to left. We also have two polarization states from vertical to horizontal. " The goal is to get detailed texture maps so the rendered face can have the same lighting as the static images. "So we analyze the specular reflections, the shine, to get high-resolution skin detail, Debevec says. "We can separate specular " 10 ■ CGW Sep t em ber / O c t ober 2013 from subsurface scattering, which are the two main components in how faces reflect light – the subsurface diffuses the light and the specular light plays off the surface. " The high-resolution scans produce geometry that is accurate to one-tenth of a millimeter. "We get the resolution of the face down to the level of skin pores and fine creases, Debevec says. " Recently, the group began shooting high-definition video from several points of view using Canon EOS-1 DX cameras and created a markerless facial motion-capture technique. "What we do is similar to what Mova was doing by using florescent makeup to track facial textures, Debevec says. " "In 2006, the resolution of good, digital still cameras wasn't enough to see facial texture. But now, we can see the bumps and divots, the skin pores, freckles, and other imperfections ■ FROM LEFT TO RIGHT: Geometry and reflectance data from multiple static scans are combined at USC/ICT with dynamic video frames to recover animated high-resolution geometry that can be relit under novel illumination. well enough to track from frame to frame and see how the skin is moving. " At SIGGRAPH, the group demonstrated its work with a realtime performance of "Digital Ira. Graham Fyffe, an ICT com" puter scientist, explains: "We used LightStage to capture the aesthetic expressions using digital cameras, then we captured a video of Ira's performance. We track movement of his face based on the high-definition video. The look comes from the expression scans in LightStage. The LightStage capture gives us the range of expressions the actor can make with his face, and the scans inform the tracking by providing almost a library of what the person's face can do. " The tracking and retargeting software is proprietary. "The main thing we're trying to do is improve the way digital characters move, especially their faces, Fyffe says. "We want them " to look real through a wide range of expressions. This is not a solved problem. It took a great deal of artistic effort to achieve the kind of performance we saw in Benjamin Button. We're trying to build something that captures an actor's performance more directly with less effort on the part of the artists. We're trying to push the quality forward, and we think we have done that by combining multiple scans with video in a way people haven't done before. "