Computer Graphics World

n n n n Motion Capture Giant crews surveyed the set and registered it to the environment in MotionBuilder, to align the two boxing rings—real and virtual— exactly. This helped Giant determine origin points for the live-action camera and the ro- bots so that the CG robots' feet, for example, would look like they were on the floor of the real ring. To capture the movement of the live- action camera, Giant placed and calibrated motion-capture cameras around the stadium to create a usable capture volume. "We had to make sure the calibration was exact between the virtual fight performed at Giant and the real set," Schatz says. "Some- times that meant offsetting the real-time stream from the live-action camera." Prior to filming, Giant had rendered the robots in a void, with an alpha channel for compositing in the Simulcam system. So, dur- ing filming, wherever the camera moved, the operator would see an appropriate view of the CG boxers in the camera's viewfinder overlaid on the live-action stream coming into the camera. In other words, the camera operator would see the robots as if they were in the ring fighting, and at the same time, see everything around them in the location—the actors, the crowds, the ring, full-scale mock-ups of the ro- Crowd Control Extras on location filled only the first few rows of the arenas in which the robots boxed. So, to fill the empty seats, digital effects supervisor Swen Gillberg at Digi- tal Domain led a team that created an innovative crowd-replication system using 2D cards. The crew calls the system "Swen's Kids." Gillberg had 80 extras to work with, so he shot five positions for each using three Sony EX3 HD, 24-frame progressive cameras: front on, quarter front, quar- ter profile, three-quarters back, and back. He shot them from camera level, from on high looking down, looking up, looking severely up, and in a medium shot. "Then for each of the five positions, we had them do four different emotions for five seconds without cutting the camera," Gillberg says. "We ended up with 8TB of extra data, but we covered all the angles you'd want in an arena." Gillberg and his crew then put each of the extras on individual cards. "We could have a crowd of 40,000 people, or we could push in to four people," Gillberg says. "We had enough information to do three extras edge-to-edge on frame, and they would hold up at 2K." Digital environments supervisor Geoffrey Baumann and digital environments lead Justin van der Lek created the system that organized and oriented all the cards. "It would have all the cards facing the camera in [The Foundry's] Nuke," Gillberg says. "We would bring in the shot camera, and the system would cal- culate how many cards would be in the frustum of the camera. It would figure out which cards to grab based on where the camera was—the back of people's heads, their faces, whatever. Then, it would calculate out how to render all those people in manageable chunks, render them out, and then combine them as a post process. The genius part was giving non-technical artists sliders they could use to get emotional beats." –Barbara Robertson 22 October/November 2011 bots from Legacy outside the ring, stunt actors on stilts wearing image-based capture (IBC) "jammies," everything. It was a real-time com- posite against a live-action background. "The goal was to make the experience as in- tuitive for the first unit as we could," Madden says. "Because, think about the alternative. What would the camera operator do? Whip the camera around and then in postvis hope you could put robots in the right place?" The robots that were fed into the camera could box in fights that extended beyond the performances selected for the previs. "If we had a 20-second take, we'd keep the whole 20 seconds, even if we had used only two three- second pieces for the previs," Madden says. "That way [the camera operator] could shoot the action as if the robots were there." Thus, although the operator shot with the previs in mind knowing the angles Levy had chosen and how to get from one shot to the next, because he had the entire take, his choices on the day of the shoot could be intui- tive. As he moved the camera, he could see the robots as if they were really on stage, and he could respond to the action authentically. "He could walk to the corner of the ring and what he saw would be correct," Schatz explains. It wasn't so obvious for the audience. Dur- ing the rehearsal and later, when filming began, Levy gave feedback to the crowd, as if he were a radio announcer, to help them understand what was happening in the fight. "We had ideas about sophisticated tricks that would help the 100-plus people around the ring feel con- nected to the fight, but what it came down to was having the robot performers who did the fights repeat them in the ring, and have Shawn doing voice-over of the fight like a sportscaster," Madden says. Even though they couldn't see anything in the ring, Levy's narrative would re- mind them of the rehearsal. If the action moved to one corner, they'd know to look there. They'd know when to cheer and when to boo. "We had never done Simulcam for move- ment this frenetic, for such a fast-moving cam- era," Madden says. "In the beginning, we were nervous. We couldn't have any significant de- lay in the CG robots displayed against the live action because that would be too distracting for the camera operator." Madden describes the first take: "Dave Em- merichs, the camera operator, was on a Steadi- cam rig for our very first take. We had the virtual robots composited into his viewfinder, so he watched the beat we wanted to shoot. He talked with Shawn. He went into the ring. Shawn called 'Action.' There were hundreds of extras jumping up and down, beer flying, people bumping the other camera we used to cover the shot. It was a classic, hardcore, live- action scenario. And, Dave covered it beauti- fully. We knew then that it would pay off like we had hoped." The resulting plate became the first temp version of the shot. "As soon as we shot a plate, we had robots in it," Nash says. "And the edi- tor had that to cut with." Outside the ring, robots came and went, and waited to fight, and sometimes these robots interacted with the actors and ex- tras. "There's a shot with Noisy Boy walking through the crowd with Charlie and Max that wouldn't have been practical to do at the stu- dio because the actors drove the shot, the way they walked through the crowd and navigated the robot," Madden says. "So we did the shot with the robot performer on stilts walking be- side Charlie and Max." Similarly, stunt actors on stilts performed other robots, each performer wearing an IBC suit, so-called "jammies" that matched their character's personality. "We put Noisy Boy in purple, Midas in yellow, and Atom in gray," Madden says. "Each suit had markers in col- ors that would stand out in particular light- ing conditions."

• Cover