Issue link: https://digital.copcomm.com/i/1011494
e d i t i o n 3 , 2 0 1 8 | c g w 1 3 animation within the game environment. More recently, Epic again partnered with Cubic Motion and 3Lateral to continue to advance realistic human performance in real time with the introduction of Siren, the lat- est lifelike digital actor and the star of Epic's technology demo of the same name, which was shown on stage during GDC 2018. "When we began working on 'Siren,' we knew from the beginning that it was going to push several boundaries," says Libreri, who had extensive experience in visual effects for feature films, including Star Wars: The Force Awakens, The Matrix trilogy, War of the World, and more, before he began focusing on real-time technology. The same holds true for Andrew Harris, studio CG supervisor at Epic, enabling him to look at real time from a pre-rendered perspective, as well. "In the industry, there's been an interesting shi from pre-rendered entertainment to the real-time medium because of the increase in fidelity in motion capture. It's an extremely exciting time right now," he says. "I'm one of a growing group of migrants from the VFX industry who is coming into the game space and trying to discover what's possible with these tools." Harris sees "Siren" as an advancement of the work done for Senua's Sacrifice, albeit an evolutionary one. "In some sense, a lot of what we're exploring in real time right now is already ground that's been covered in visual effects work that's pre-rendered, when you're afforded 10 hours a frame," he says. "What's really groundbreaking is that we're trying to employ the same techniques in real time, so our render times are 42 milliseconds per frame, rather than hours per frame." Also, Senua was an artistically created character, though it was based on scanned data, whereas Siren is intended to look pre- cisely like the actress on which the character is based. But with Siren and even Mike from "MeetMike," as well as some recent work with Andy Serkis, "we were as precise as possible with the capture data. Every detail of the asset needs to hold up in any lighting condition and from any angle," Harris says. IN HIGH FIDELITY Epic began working on the "Siren" project a little over a year ago, which began as a collab- oration between Epic and Tencent, with the goal of creating a proof-of-concept demon- stration to show the capabilities of Epic's Un- real Engine 4 and what the next generation of digital characters would look like. Once again, the group turned to 3Lateral, which digitally scanned Chinese actress Bingjie Jiang, whose likeness would be reflected in the CG Siren model. Jiang was photographed extensively, with 3Lateral scanning her entire body as well as her face while performing various FACS poses in order to isolate each muscle movement. "It's not that we captured just a single pose; she does a range of motions, and we cap- tured a 3D model in all of those positions," Harris explains. When 3Lateral is building the face, the rig actually blends in a new shape for every expression. "When the lip curls, we have a lip curl model that we're blending in," he adds. "Much of the complexity of the rig is in terms of managing those shapes and how they blend from one to the other." All told, 3Lateral provided 3D and 4D scans, along with materials "that were as precise and as close to the true-world values as possible," Harris notes. "We create digital models that are accurate down to the pore level. We also come away with enough photographic imagery to create all of the textures that go into making the character look realistic." Alas, with high-fidelity data comes large file sizes, "larger than we had ever dealt with before in-engine," Harris notes. "To bring Siren into the engine, we were working with an FBX file that was about 4GB." While 3Lateral worked on the facial mod- el, Epic and Tencent refined the body scans, optimizing the topology of the dense scans. Following the scanning process, Epic re- ceived a model that was an extremely accu- rate representation of Jiang's body, though it required Epic to do a little cleanup and make a few adjustments. For this work, the artists sculpted the model using Pixologic's ZBrush and re-topologized it in Autodesk's Maya, before the rigging team got to work in Maya. "This way we have an exact copy of the entire rig in-engine so we could have a one- to-one match between the animations and deformations in Maya and the animations and deformations we'd get in the engine," Harris notes. MAKING MOVES Re-creating the subtle intricacies of move- ment can be the difference between a real- istic digital re-creation and one that leaves audiences feeling unnerved. So, it was vital that Jiang's movements aligned perfectly to the digital model. Jiang was outfitted with a motion-capture suit and head-mounted camera, while Epic and Tencent utilized Vicon's Vantage optical mocap system, its Shögun soware, and VUE video cameras to capture precise and authentic movement and to add the character animations over the reference footage in real time.