Issue link: http://digital.copcomm.com/i/371911
j u ly . a u g u s t 2 0 1 4 c g w 1 1 "Because Muse is new, we had between six and eight people from R&D helping with the facial solves and with the pipeline," Weaver says. "Since the R&D people were making the system better as we went along, we decided they might as well be part of the process." Prior to the on-set motion capture, the team had the actors perform expressions while being scanned with Disney Research's Medusa system. The Muse team decomposed those scans into components within Fez, ILM's facial animation system that animators used to re-create the expressions. A er facial motion capture, layout and R&D teams triangulated the markers in the 2D images into 3D space, frame by frame, and then applied them to controls on those components in Fez. "We also ran the data through a contour tracker to track eyelid, inner lip lines, and pupils," says Tim Harrington, animation su- pervisor. "All that data drives the face." At the end of the process, animators work with the same cuts with lines of dialog and associated video, and we'd see faces jumping from line to line. We had to blend between them. Because the way Muse works, we could copy and paste keyframes of motion data and the an- imators could massage the in-betweens. This wasn't possible before. Now, it's not only possible, it's easy." Each actor wearing the head-mounted cameras had 128 dots painted on his face. At ILM, layout artists calcu- lated the position of those dots in 3D space in each frame by using the cameras' positions relative to the hel- met and the dots captured on the le and right sides of the actor's face. The Muse team applied those dots to the facial rig in Fez. Then, they fi tted the dense data from the Medusa scan onto the 128 points. "Medusa gives us some arbitrarily high number of points, maybe a mil- lion points per frame, for high-quality reproductions of the skin," Koperwas says. "It tells us where the wrin- kles are. We fi t that data onto the Fez rig." When dots captured during the actors' perfor- mance move, the face moves – but not directly. And this is a key diff erence between Muse and previous systems. "We fi t a set of controls," Koperwas says. "As the dots move, we translate that into a set of controls the animator can edit. Previously, we'd use the sparse dot pattern to de- form the face, which gave us nice results, but it was diffi cult to edit. Basically, we were working with an animated sculpt, a diff erent shape on each frame. We could add to it or remove something, but we couldn't really change it unless we re-sculpted it." Now, the data is on controls, on slid- ers instead, and animators can easily edit the motion. The other key diff erence is that the data came from two cameras, not one. "You can fi t the 2D data from one camera to a facial ani- mation rig, but it isn't as 3D accurate," Koperwas says. "The slightest diff erence can have a big impact." The team felt the impact of the Muse system in sev- eral ways. "It gave us the fl exibility to stitch together diff erent pieces, from those cap- tured in broad daylight on location to ones captured on a soundstage and have the same consistent quality of solve," Koperwas says. "That gave the fi lm- makers and animators the freedom to work as they wanted. They could fi nd the best timing and the best moments from characters in diff erent shots, and we could translate that into a system that didn't give animators aneurysms." – Barbara Robertson ACTOR MEGAN FOX WITH THE FOUR TURTLE ACTORS WEARING FACIAL MOCAP RIGS AND ILM'S DELTA SUITS. ILM'S IMOCAP SYSTEM TRANSLATES THE SUITS' PRINTED MARKERS INTO MOTION DATA.