Post Magazine

January/February 2020

Issue link: http://digital.copcomm.com/i/1208556

Contents of this Issue

Navigation

Page 33 of 43

www.postmagazine.com 32 POST JAN/FEB 2020 the model on a per-frame basis to match the actor's performance. "Then, we retarget this performance to a younger version of the model and render it through lighting and texture," Helman says. "We had no keyframe animation in this project at all. We didn't want to change the performance." Mo d e l s A n d Tex t u re s A team led by digital model supervisor Paul Giacoppo sculpted contemporary and youthful models of each actor, changing the geometry in the chin and neck as needed. "Each model started as an accurate scan of the actor at his current age using a combination of [Disney Research's] Medusa for likenesses and facial expressions, and Otoy for facial detail," Giacoppo says. "Then, by looking at past films, we sculpted younger faces. We had a slider that could take us from current ages to previous ages." The models provided the form and larger-scale bumps and pores. Texture artists, led by supervisor Jean Bolte, added the finer details, working from the Otoy scans and photographic reference. "We de-aged them in stages," Bolte says. "Each stage had to have wrinkles and age spots painted out judiciously as we figured out how much to take away. I'd look pixel by pixel, zooming in to make sure of the integrity. We didn't want them to be too pretty. We wanted to keep things a makeup artist might have taken out. I was well aware that we could have ruined the movie if we didn't nail it." She smiles and says, "I think we pretty much nailed it." For what Helman calls a "sanity check," the crew spent two years gathering a library of performances for the three actors at the targeted ages from different movies. An AI-based program they've dubbed Face Finder found frames to match rendered frames in terms of age expression, pose, camera angle and lighting. But, they weren't aiming to exactly match the actors at those ages. "Martin said the he didn't want us to take DeNiro from Taxi," Giacoppo says. "He had to be the young- er self of the character he was playing in this movie. A young Frank Sheeran." Adds Bolte, "We didn't have a clear goal. Not only do these actors appear different from one film to another, they're different even from one shot to another. It was a matter of, who is this character Frank Sheeran? We had to find that. I spent days studying images of DeNiro. He can change his ex- pression with the raise of an eyebrow. He is a master at being a chameleon." All told, the crew spent nine post production months working on the 1,750 shots, but that was after they'd spent four years making the post pro- duction possible. "We knew the risk we were taking," Helman says. "So, we invested those four years of development to have something completely performance-driven. If you follow the natural progression of visual effects, you find that the next thing is markerless perfor- mance capture. "I remember Ewan McGregor's reaction when he walked onto a bluescreen stage," Helman continues. "He said, 'What the [expletive] is this?' I thought an actor could imagine the set. But, after working with these actors on The Irishman, I realized what he meant. It's crucial for actors to be where they're meant to be." G e m i n i Ma n : We t a D i g i t a l In this film, 50-year-old actor Will Smith plays both the character Henry Brogan, an aging former Marine Scout sniper now an assassin, and Junior, a clone of Henry at about age 25, who is also an assassin. In some scenes, the two are face to face. Directed by Ang Lee, the action thriller received mixed reviews — with Rotten Tomatoes' aggregate scores for critics a rotten 26 percent approval, but audience scores at a fresh 83 percent. Lee chose to shoot the film in 4K 3D at 120 frames per second, which bothered some critics and made life difficult for Weta Digital's rendering team. Bill Westenhofer, who had received an Oscar for Lee's Life of Pi (and another for The Golden Compass), was the overall visual effects supervisor; three-time Oscar nominee Guy Williams supervised the visual effects created at Weta Digital. To create the clone of Will Smith's character at age 25, the Weta Digital crew adopted what has become a traditional method of capturing perfor- mances, one this studio honed especially through the three award-winning Planet of the Apes films. To devise Junior's look, however, researchers at Weta Digital developed state-of-the art technology for skin color and textures. On set, when both characters appeared in a scene, Smith's stand-in, Victor Hugo, played Junior, knowing he would be replaced later. Then, Smith played Junior in the same shots wearing mo- tion-capture "pajamas" and a head-capture rig. In the film, Junior is 100 percent digital. "A performance doesn't exist only in the face," Williams says. "Everything you do moves you, from your feet to your eyes, and all of this adds to how we recognize a person. It isn't solely about facial motion. So, we choreographed all the motion to- gether. Otherwise, you end up with a bobble head." The face carries most of the emotion, though, and Williams notes two challenges in creating and performing Junior's face: "First, it becomes easy to lose his likeness," he says. "And second, Will Smith hasn't aged much in 25 years. We had to get into the deep science of youth versus age to create enough of a difference. We knew the distance from lips to nose changes, and the jowls and cheeks sag. But, we also had to put youth in his pores, in the color around his eyes, in the moisture of his lips and in his eyes to make sure everything our brains perceive as youth is properly represented. Digital humans live or fail in insanely-subtle nuances. Skin turned out to be a major component." Po r i n g Ove r De t a i l s Weta Digital started by creating a digital model of Will Smith at his current age using photo shoots, photogrammetry scans, skin lighting capture at ICT and two FACS sessions. Then, they modified the digital model of 50-year-old Smith to change his facial structure and appearance. For reference, they had Smith's early films and 23-year-old actor Chase Anthony, whose skin looked like Smith's. Initially, the crew considered relying on their standard approach in which they use a live cast for skin textures. "But, one of our shader guys thought he could grow the pores," Williams says. "Early tests gave us hope, and in the end, he created a pore structure that was better than anything we'd have gotten from the live scans. It's not 100 percent accurate, but it's incredibly accurate. If Will Smith had 35 pores in an area, we might have 34." The simulation is controlled with empirical maps that define how to grow the pores — deeper here, isotropic there, denser, sparser and so forth. "What happens is that we 'pelt' a number of points to distribute the points evenly across the surface, and then flow the points across the face," Williams explains. "From every point, the simulation draws lines to neighboring points without crossing another line. The software interprets the flow field and can take a bias from the flow. That creates anisotropy: The flow of pores in one direction might be more dominant than in another direction." The simulated pore structure resulted in a nine-million tetrahedron mesh. "The beauty of this is that we can move it," Williams says. "We can pipe the facial animation into the simulation software with the mesh. The mesh moves based on motion capture cleaned up by an animator. The way the face moves shapes the pores and changes the shape of highlights in an anis- trophic way. We can get micro-wrinkling; the pores can collapse into fine wrinkles." Thus, the simulated pore structure provided the model for skin texture. For color, the crew simulated melanin and blood flow. Rather than painting multi- ple color maps, they first created pale-pink skin us- ing blood flow under the surface, and then layered in two types of melanin to color Junior's skin. As a result, the color of Junior's face comes from a complex interaction of simulated melanin and blood flow with light. It doesn't depend on colored light bouncing off a textured surface. Mo re T h a n S k i n De e p "Melanin is a pigment layer with thickness," Williams says. "The density creates the color, and it's an- gle-dependent. At an angle, you see more of the thickness, so it looks darker than when you view it straight on. We would squeeze blood and melanin

Articles in this issue

Links on this page

Archives of this issue

view archives of Post Magazine - January/February 2020