Issue link: https://digital.copcomm.com/i/589428
is expected to play a big role in virtual and augmented reality as more applications in this realm spill forward. Facial capture along with hand/finger capture has led to the evolution of "perfor- mance capture," whereby an actor's body movements and facial expressions are acquired and translated onto a digital character. Robert Zemeckis's now-defunct ImageMovers Digital evolved its moviemaking process around performance capture in CG films such as The Polar Express, Beowulf, and A Christmas Carol. There are many examples of successful performance capture, and there is one person who has perfected the art and helped propel it to a new level. That is Actor Andy Serkis, whose movements and expres- sions have been applied to sev- eral memorable CG characters such as Gollum, Caesar, and King Kong – with Weta's tech- nological know-how, of course. In these cases, Serkis's acting skills were captured, trans- lated, and applied to the CG character. The performances were so successful that many had discussed whether the performance of Gollum should lead to a Best Actor nomination for Serkis. M O V I N G M O C A P F O R W A R D Like facial capture, full-body motion capture has evolved by leaps and bounds. While mo- cap's roots were as an analysis tool in the fields of science, sports, and education, it is well known for its use in the video game Prince of Persia. Similarly, the film Sinbad: Beyond the Veil of Mists stakes its claim as one of the first movies made primarily with motion capture, an optical system whereby the actor was covered with tiny reflective tracker balls whose movements were triangulated within a computer. Initially expensive and time-consuming, motion capture was first used as a specialized tool, oen getting the performance "80 to 90 percent there," but that last 10 to 20 percent was extremely difficult to tackle. For those who know their CG history, you will recall that optical capture also was used in The Mummy, for the crowds in Gladiator, and for the first main, fully digital character Jar-Jar in Star Wars: Episode I – The Phantom Menace. Although Final Fantasy: The Spirits Within tanked at the box office, the film set a benchmark for the technology, with more than 1,300 live-action scenes that were turned into CG animation. Yet, it was film legend Peter Jackson whose visions set mo- cap on a meteoric rise. When Gollum became a main charac- ter in The Lord of the Rings: The Two Towers (2002), he did so thanks to a new advancement: real-time performance capture that could be achieved on set with other actors "in the mo- ment". Well, almost. First, Serkis performed the scenes on set with the other actors, and then the same scenes were done without Serkis. Later, Serkis would replicate his performance back at Weta while wearing a mocap suit with markers. For The Two Towers, Gol- lum's facial expressions were hand-animated, as animators used reference footage of Serkis as a guideline. Not so for Jackson and Weta's King Kong, for which tiny markers were attached to Serkis's face to track subtle muscle movement. This process was used for other well-known digital characters, including Davy Jones in Pirates of the Caribbean: Dead Man's Chest, but in that instance, ac- tor Bill Nighy's full performance was filmed on set, not separate- ly on a mocap stage. Avatar, Happy Feet, and The Adventures of Tintin proved that mocap could successfully transform a digital character into one that is very human- like, one that can dance, and one that can act. But Rise of the Planet of the Apes (2011) became an evolution point for the technology, with Serkis transforming Caesar into a lead digital character like no other. Weta had experimented with on-location motion capture for The Lord of the Rings and had pushed the state of the art for the performances in Avatar, but what the studio faced with Rise went far beyond those requirements. That is, until the group topped that for the sequel Dawn of the Planet of the Apes (2014), for which they used a new mocap system that enabled them to gather perfor- mance-capture data in extreme locations and weather condi- tions – ideal, since this marked the first time Weta filmed entirely on location (sometimes in conditions that were less than ideal). The sequel also entailed Weta capturing the actors' performances on loca- tion during filming using mocap cameras, helmet-mounted facial rigs, and spots of LEDs on the actors' bodies. However, the new system was wireless, which could be placed freely around the set. Since then, with movies like The Avengers and The Hobbit series, performance capture has continued to evolve, as the equipment has become more durable and can sustain the physicality of the actors on set, while the fidelity enabled by the hardware has reached new heights. And let's not forget about the soware, which has simplified the process and made the technology attractive for just about everyone. M O V I N G F O R W A R D Without question, the applica- tions and the equipment have evolved tremendously over the years. Early low-frequency magnetic systems tethered performers, thus limiting their range of motion. Mechanical systems, which measure joint angles, are also limiting in terms of motion but can be used in any environment. Optical systems use markers, which are usually tracked by infrared cameras. This continues to be a very popular method, though occlusion can be an issue. There are active (pulsed- LED) markers, passive (reflec- tive) markers, and semi-passive markers. There are reflective and optical systems. And there are markerless systems, inertia systems, and magnetic systems. Each offers advantag- es and disadvantages, as does motion capture itself. There are expensive high-fidelity systems and low-cost options. Many systems require Lycra form- fitting suits with embedded or attached markers, while others are marker-free. Motion-capture options come in many flavors to suit a variety of needs. Today, motion capture – or the more-encom- passing performance capture – is a vital animation tool, whether for background characters, crowds, lead digital characters, characters that are humanlike, and those that are not – thanks to the technologists and visionaries that developed the systems and daringly put them to use in new, unique ways. ■ Karen Moltenbrey is the chief editor for Computer Graphics World. s e p t e m b e r . o c t o b e r 2 0 1 5 c g w 2 1