Computer Graphics World

Winter 2019

Issue link: https://digital.copcomm.com/i/1192296

Contents of this Issue

Navigation

Page 19 of 47

16 cgw w i n t e r 2 0 1 9 The rig has three cameras placed side by side on a 30-inch bar, narrow enough to fit through a door. In the center is the RGB camera, and on either side are two infrared film-grade Arri Alexa Minis. "We needed to neutralize the light without changing the lighting on the set," Helman says, explaining the need for the infrared cameras. "In effect, rather than taking the actor into a controlled environment, we cre- ated a controlled environment on the set." No other "witness" cameras were needed. The infrared light didn't interfere with the theatrical lighting on set and produced images without shadows. The actors and director didn't see it. The actors could sit at a table in a crowd- ed, busy restaurant and lean toward each other to talk. Scorsese could film them in close-ups, and as he did, ILM captured their facial expressions using the three cameras on that one rig. Two camera operators controlled the cameras remotely. One operator managed the main camera. Another operator con- trolled the infrared cameras, which have a different depth of field. No Keyframe Animation Then, the magic happened. Helman describes the process used to create DeNiro's more youthful Sheeran from footage and data captured from the three cameras. "Once I got the take, I brought the foot- age from the three cameras here to ILM," Helman says. "We also had the data gath- ered on set: HDRIs for light and density, and Lidar data to know where all the lights were. The footage went through layout to solve the camera [determine the camera view], and we did matchimation for the bodies and heads. All that data – the layout, roto, HDRI, Lidar – went into Flux with information from the three cameras. The soware made a cocktail of it. Flux figures out where the ac- tor is in 3D space and derives geometry from the three cameras to create a digital double of the actor." Flux produces an albedo model show- ing a representation of light and textures and a plastic shaded render. The soware then compares its digital double to a model built of the actor and deforms the model on a per-frame basis to match the actor's performance. "Then, we retarget this performance to a younger version of the model and render it through lighting and texture," Helman says. "We had no keyframe animation in this project at all. We didn't want to change the performance." Models and Textures A team led by Digital Model Supervisor Paul Giacoppo sculpted contemporary and youthful models of each actor, changing the geometry in the chin and neck as needed. "Each model started as an accurate scan of the actor at his current age using a com- bination of [Disney Research's] Medusa for likenesses and facial expressions and Otoy for facial detail," Giacoppo says. "Then, by looking at past films, we sculpted younger faces. We had a slider that could take us from current ages to previous ages." The models provided the form and larger- scale bumps and pores. Texture artists led by supervisor Jean Bolte added the finer details, working from the Otoy scans and photographic reference. "We de-aged them in stages," Bolte says. "Each stage had to have wrinkles and age spots painted out judiciously as we figured out how much to take away. I'd look pixel by pixel, zooming in to make sure of the integrity. We didn't want them to be too pretty. We wanted to keep things a makeup artist might have taken out. I was well aware that we could have ruined the movie if we didn't nail it." She smiles and says, "I think we pretty much nailed it." For what Helman calls a "sanity check," the crew spent two years gathering a library of performances for the three actors at the targeted ages from different movies. An AI- based program they've dubbed Face Finder found frames to match rendered frames in terms of age, expression, pose, camera angle, and lighting. But, they weren't aiming to exactly match the actors at those ages. "Martin said he didn't want us to take DeNiro from Taxi," Giacoppo says. "He had to be the younger self of the charac- ter he was playing in this movie. A young Frank Sheeran." Adds Bolte, "We didn't have a clear goal. Not only do these actors appear different from one film to another, they're differ- ent even from one shot to another. It was a matter of, who is this character Frank Sheeran? We had to find that. I spent days studying images of DeNiro. He can change his expression with the raise of an eyebrow. He is a master at being a chameleon." PESCI, 76, AT LEFT; YOUNG BUFALINO AT RIGHT. HIS FACE HAS NO MARKERS. AN RGB CAMERA AND TWO INFRARED CAMERAS CAPTURED FOOTAGE USED BY FLUX.

Articles in this issue

Archives of this issue

view archives of Computer Graphics World - Winter 2019