Issue link: https://digital.copcomm.com/i/42006
Motion Capture n n n n tensions, which were foot-long little crutches, helped. Even though LEDs didn't extend into the crutches, the system knew how far the crutches extended beyond the hand. "It's procedural, but not automatic," Lem- mon explains. "We had to adapt and adjust it depending on what the actor was doing, whether he was standing, sitting, or on all fours pushing with his knuckles and legs." Animators then refined the performance using keyframe animation. "We still have lots of keyframe animation, Actors wore arm extensions on set, and although the LED markers didn't extend the full length, the data translation system took the "crutches" into account. actors touched an ape actor, it might give away the size." Talk to the Animals The data captured from the actors' perfor- mances moved from the stage to a department of motion editors who translated that data into curves for an animation system, using techniques originally developed for King Kong and refined since. The goal was to move data representing the captured motion of an actor's skin onto CG muscles beneath the digital face. Thus, when an animator moved a muscle in the animation system to create an expression, the digital skin behaved appropriately. Dedicated actors played all the principal characters, and to aid with the data transla- tion for those actors' performances, the crew at Weta had "pre-calibrated" each actor. "Our facial system is built on Paul Ekman's FACS system," Lemmon says. "We had them go through a standard battery of FACS expres- sions and a range of movement for their body. We filmed them with witness cameras and digital SLRs. For the hero actors, we also did a high-resolution motion-capture session using tiny capture markers on their faces." Ultimately, though, the data would need to drive simian characters, not humans. "The biggest difference was in the design of the faces," Lemmon says. "The Na'vi in Avatar mimicked the actor's features. And, the art di- rectors redesigned Gollum's face to look more like Andy Serkis, which helped give us solid anchors for markers. It is more difficult with apes. There isn't a one-to-one correspondence. We had to look at the expression Andy made and translate it into Caesar's anatomy." Before creating Caesar and the other apes, the crew did extensive research into ape physi- ology, muscles, facial structure, and body lan- guage. "As you can imagine, in addition to the differences in anatomy, there are also dif- ferences in the way chimps use facial expres- sions," Lemmon says. "When a chimp smiles, it can mean it is scared or it is trying to be aggressive." Even though the film is an origin story, and the animators and artists tried to be as faith- ful to modern-day chimpanzees, orangutans, and other apes as possible, they knew that movie audiences would not understand that a chimp's smile doesn't mean it's happy. "We looked at reality as a reference point as much as possible," Lemmon says. "But when it came to Caesar's facial performance, we wanted Andy [Serkis'] performance to come through and be readable. So, we set up a system that translated the movements of Andy's facial markers into muscle firings that we translated into something slightly differ- ent, but with the character of Andy's perfor- mance, for the ape. But this happens when we translate performances onto humans, as well. We need a lot of human intervention to get the result to look as close as possible to the facial reference we get from the witness cameras." Big Steps To help with the facial and performance animation, Weta Digital researchers created a new muscle system. "We have a dynamic solve that goes on top of the animation layer," Letteri says. "It adds ballistics to the face, and smoothes the muscle and skin movements across the face, distributing them properly." For the bodies, a character-mapping pro- cess scaled the data captured from the human actors to the size of the ape they play, and adapted the proportions appropriately for the apes' longer arms and shorter legs. The arm ex- even with the performance capture," Letteri says. "You can work out the best translation you can think of through the software, and still someone has to make adjustments to the leg, heel, and knee to get the weight right." Animators also performed many of the background characters, and for shots that the human actors couldn't perform, animators keyframed those characters as well. "People can't climb a tree like an ape, no matter how hard they try," Letteri says. As with most new techniques and technol- ogy, Weta Digital adapted previous systems to create the apes' performances for this film. However sometimes, evolutions are revolu- tionary. It wasn't important to capture actors playing Avatar's Na'vi outside in the real world because they lived on another planet that was a digital environment, but the apes in Rise of the Planet of the Apes needed to be in the real world. When King Kong was in the real world, Serkis was off to the side in a scissor lift to give the actors a correct eye line. The motion capture happened later, on a separate stage, where Serkis performed to the live-action foot- age. "We could have done that for this film because Andy is so great," Letteri says. "But, we didn't want the actors in this film to act to tennis balls." Letteri believes the solution they devised —a flexible, portable adaptation of the perfor- mance-capture system, which can capture actors' facial and body movements in day- light, on location—is a big filmmaking break- through. "The whole point is to make the visual effects part of the filmmaking process," Let- teri says. "That's what Jim [Cameron] was after with Avatar. Breaking down the bar- rier between live action and digital film- making. It didn't make sense for Avatar, but it did for this film. This is the last step." n Barbara Robertson is an award-winning writer and a contributing editor for be reached at BarbaraRR@comcast.net. August/September 2011 15 . She can Computer Graphics W orld