Issue link: http://digital.copcomm.com/i/1399888
8 cgw j u ly • a u g u s t • s e p t e m b e r 2 0 2 1 Douglas Douglas is prime example of the current state- of-the-art in the creation of digital humans. Douglas is a realistic autonomous digital human whose sophisticated AI enables him to not only interact with humans, but react in real time, and is capable of learning and recalling information insofar as he can carry on a focused conversation by paying attention to what is being said and formulating an appro- priate response. Douglas is based on an actual human, Doug Roble, and is the "Adam" of Digital Domain's Digital Human Group (DHG). The group want- ed to create the digital human based on a real person, to see how closely they could replicate the person; they also needed easy access to the subject. Roble, senior director of soware R&D there, fit the bill. Digital Douglas is being developed as a per- sonal assistant, and appears on-screen during video chats. "We want Douglas to be as fast [with his responses] as a real person," says Roble. "And that has been one of the big challenges right off the bat. It's all about speed. He has to move quickly, talk quickly, and he has to respond quickly so that his feels like a real conversation." Machine learning is teaching Douglas that a pause in speech oen means it's his turn to talk. But, sometimes the other person is just catching his or her breath, so more fine-tuning is needed in this area. "Overall, he's quick, but not as quick as we want him to be. We're always working on mak- ing him faster and faster," says Roble. While you are speaking to Douglas, he is examining your face to see if he recognizes you (through facial recognition), and if so, ac- cesses any memories he has associated with you — prior to any memory reset. (DHG is well aware of potential privacy issues that could arise should they ever release an application to the public.) Douglas also sends your speech to a speech recognizer, which then forwards the text of that conversation to three different natural language processing (NLP) conver- sational agents that come up with different responses — those that are purely conversa- tional, some that are more scripted, and others that are more factual-based (by searching the Internet). And then he decides which of those three response types is more appropriate. Semantic and emotional analyses of the text determine the best emotional range for Douglas's response (happy, sad) and then generate his voice, while creating an appro- priate facial performance that coincides with that determination. An emotional range for the body is also calculated. "All that is pulled together and rendered out," says Darren Hendler, DHG director. In ad- dition, real-time cloth dynamics are calculated. "Now I run that through a separate process that does the neural rendering on top to make Douglas seem more realistic than he does straight out Unreal, and I deliver that into a Zoom call, for instance. And that is happening in every frame in real time. It's fast and amaz- ing, but not fast enough to be a real person." Hendler continues: "We come from a back- ground where the single frame of an image could take five hours to render for a feature film. So, this is mind-blowing for us to be able to do all of that as fast as we do." And the power behind this feat? "Nothing you can't buy from a store," says Hendler. Douglas does not need a supercomputer; typically, the application runs on two separate computers — one devoted to rendering and another devoted to speech, AI, and so forth — since many of the functions need to run in par- allel. Although, rather than two computers, it can be run on a single machine with two GPUs. And because it runs in parallel, some functions can be turned off if desired. BUILDING A BETTER DOUGLAS Which aspects of Douglas still need improve- ment? "Just about everything" responds Ro- ble. Well, everything that stops a person from thinking Douglas is a real human. "We don't think Douglas looks photoreal. Rather, we are saying, 'look at what's possible, and look at the trajectory of where this is going and where this will be,'" Hendler says. "But look at what's going on in a few milliseconds; it's quite amazing." Amazing, indeed. During a Zoom call ap- pearance, for instance, when he is quiet and moving ever so slightly, many (including this editor) do a double take, wondering whether or not Douglas is real. To truly simulate a real person, says Roble, will require improvement in the current technology. While machine learning can mimic human behavior, to have an application that really understands what is actually occurring in a conversation falls in the AI realm. "And that full-blown artificial intelligence is still far off," he adds, noting that the amazing examples at this time are just mimicking what they've observed. "To actually understand what's happening is really, really, really hard and doesn't exist yet." Nevertheless, there is so much more that can be done at this time — the speech synthe- sis, the expressions, the movement. "And every month Douglas gets better and better and can do more things," Roble adds. DHG's Douglas is a realistic autonomous human based on Doug Roble.