Issue link: https://digital.copcomm.com/i/1530856
LED design to what they need. Sometimes, it might not be a fully encompassing LED Volume with a ceiling. Sometimes it might just be the exterior of another building. For instance, we've used it for exteriors on 'The Batman' where the LED Volume was the Gotham skyline or the view outside of a window." One of the key aspects of virtual produc- tion is creating the illusion that the actors are in the environment on screen and not on a soundstage. The trick is to blend the phys- ical set pieces with the virtual backgrounds so they feel like an extension of the set and not just a background. "One trick to doing that is repetition. So, if the art department is building a rock for the set, we'll photo- graph or scan that rock and repeat it in the background. On 'Percy Jackson,' we were doing photorealistic environments of iconic locations, like the Metropolitan Museum of Art in New York City. We had a practical hot dog stand (that's very iconic New York City) on the Volume and we repeated that hot dog stand in the background, in the distance. We also matched the textures of the street and the sidewalk in the background to what the art department was doing, to blend the physical and virtual together," she said. For scenes inside the MET, such as the Greek room, it was important to capture every detail of the real location and recreate it in the virtual backgrounds. Here, Contre- ras's prior experience as a location artist proved invaluable. She said, "You have to pay tribute to the real location and do it right. We're doing photorealistic worlds of places that people recognize. So we go on location, do scouting, and do photography. Then, we work closely with the art depart- ment to make sure that the scale of our backgrounds for the Volume match their practical set pieces. For example, we cut off the pillars in the same place. We also adjust the lighting. We're matching the look of the environment in a way that people will recognize it, creating seamless blends and matching all the textural components of the scene." During the pre-lighting phase, Con- treras and her team will work with the DP [director of photography] on lighting the virtual environment to match the set lighting they'll be using during the shoot. "We can control the colors in the scene. We can control the lighting in the scene. We can look through the camera, look through the DP's LUT ("Look Up Table"; a tool to save color grades as a template of "pre-sets" for use on a later project), and be able to match the colors as you'd perceive them in the correct lighting through the camera," said Contreras. T h e I L M te a m u s e s n u m e ro u s D C C (digital content creation) packages to build their virtual environments. They often work in Unreal Engine 5, but the StageCraft team also uses other visual effects industry tools, such as Autodesk's Maya, SideFX's Houdini, Foundry's Nuke, and so on, to create all the needed assets. The team often renders the final pixels that will appear on the LED walls in ILM's proprietary cinema render engine called Helios. "We use whatever tool is best suited for the job. It's the same techniques that we would use, like photography. Especially for the photorealistic backgrounds in 'Percy Jackson,' we do a lot of photography. Then, we have a team that can build environ- ments, do textual paint jobs, and create and digimat backgrounds, just like we would typically do in post. The main thing to consider is that we have to make those environments work in a real-time render," said Contreras. While StageCraft will occasionally use Unreal Engine 5 during production, it's the case of one or the other, but not both. So, once the decision is made to use the Helios workflow for the production of a show, they'll stick to that. "We may do some pre-viz work in Unreal Engine, particularly for a process called VAD (virtual art depart- ment). So, we'll build a digital environment and then do scouting in VR to help the film- makers navigate and interact with the sets. We use Unreal for a lot of that. But then we'll do the actual production work in Helios," Contreras explained. O n e b e n e f i t o f u s i n g H e l i o s o v e r a commercially available render engine is that ILM's StageCraft team can modify or add features as needed. They have an in-house engineering team that can make upgrades to Helios to address produc- tion-dependent needs. "I worked on 'Thor: Love and Thunder' and we needed to create an environment that had some specific gold rendering as- pects to it. We worked with our R&D team to turn that around quickly for us. We own and maintain Helios so we can quickly build and add those bespoke features," Contreras said. "On every show that we've used Helios, we've had a bespoke need in rendering that our engineering team han- dled very quickly. So, as we are learning and growing our virtual productions — giving ourselves more challenging work — the features we add stay with the renderer. It's constantly evolving." For future virtual production artists who would have to learn StageCraft, Con- treras said not to worry. Learning the Helios render engine is very similar to learning another DCC tool like Blender, Maya, or Unreal Engine. "It's usually just translating the language. You know how to do a task using a different tool, and so the question is, 'How do I do this one thing using this tool?' We have a team of artists who can help new, incoming artists translate that knowledge to StageCraft," she said. Whether a virtual production uses a commercial render engine or a proprietary one, Contreras feels the main challenge is the same: Having enough GPU power to run everything as a real-time render. She concluded, "Sometimes you have to take out some lights, or you have to bake the light, or you have to remove some geometry to keep it running in real-time. For me, the blue sky scenario is to have bigger, better, faster computers that can process everything and allow us to build our worlds a bit larger. We're constantly setting the bar higher and higher in terms of the visuals. We want to add everything to it to make it look visually as best as possible and still have it run real-time." ■ 45 W I N T E R Q 4 I S S U E T E C H