Post Magazine

January/February 2024

Issue link: http://digital.copcomm.com/i/1516067

Contents of this Issue

Navigation

Page 32 of 39

creative workflow to reflect looks that are only limited by imagination. Yamnitsky says that generative AI within the product line will allow en- hanced generative up-res'ing of low-res- olution video to 4K, 6K and even higher resolutions, from even SD samples, and the ability to expand images beyond the original frame boundaries while main- taining resolution and colorimetry of the original, graded image. "Basically doing things with the image and removing noise, adding noise, film grain, removing motion blur, adding motion blur, scaling up, scaling down," he continues. "Removing flash photog- raphy. You can enhance details. You can sharpen. You can bring details that were not there before. So image restoration is a very, very large field that is traditionally solved by DSP digital signal processing filtering or algorithms. That goes back into the '60s and '50s. But now, better results can be achieved by machine learning and AI." How does it work? I asked just how AI works to do these miracles — it's based on a multitude of models and training from them. "So the model basically is trained on an original image and thousands of those model pairs," Yamnitsky explains. AI examines these model pairs and extrapolates needed correction based on the cleanest iteration. But doesn't this re- quire memory capacity out of the range of desktop users? "This is where the [biggest] challenge comes in, because you see, all those models require a very large footprint on your computer," says Yamnitsky. "If you install all the software that's necessary to run that, it's like gigabytes and gigabytes, where typically, a lightweight plug-in product, which is easily downloaded, installed within seconds on your machine is much smaller…Our goal is to have everything on your editing machine right there, isolated. So we streamline the data points in each filter to enough memory to handle a limited amount of AI compu- tations for each particular filter." Looking toward the future "In the future, many of the image resto- ration and cleanup tasks will basically be integrated," says Yamnitsky. "It can do things like beauty shots. It can do things like color correction, de-noising, sharp- ening — anything that makes your clips look better. For color grading, you will be able to put a model of a look you want from a film, or choose from a preselected group of images and have AI process (it), like a LUT. I cannot say it will take the place of a good colorist, but it will be pretty good. They may come from di©er- ent cameras. They may come from di©er- ent formats, shot at di©erent times of the day, with di©erent lighting. It's happen- ing. It's just a matter of development." How soon? "I think 2024 will be a very big year for AI and tools," states the BorisFX founder. "There are already titles on the market that kind of attempt to do that. And there'll be many more because the technology is out there. The algorithms are out there. It's just a matter of pro- gramming them, writing code, imple- menting them." What does this mean for the editor? "This is the beauty of it," says Yamnitsky. "If you look at Continuum. The noise filter is basically drag and drop. Noise filters usually require a lot of tweaking. But here, you drag and drop, and it knows everything. There is very little that you can control. You can probably make [it] stronger or weaker. That's about it. You play it back. You don't like something, you go back to your project, open that filter, move the slider and now render again." Text-based e¡ects "We're experimenting with text-based prompts to drive the e©ect," shares Yamnitsky. "You should be able to describe the desired outcome as text prompts, as opposed to just numerical parameters to mess with. Make the flare wider or glow purple…things like that." The future of BorisFX Over the years, BorisFX has grown through the acquisitions of compa- nies like Mocha, SynthEyes, Silhouette and Optics, corralling the market for 3D camera tracking, e©ects creation, rotoscoping and more. I had to ask Boris whether there was a grand plan, and how he chose these companies. His answer surprised me, as I thought it was the technology that attracted him. "The people," he explains. "People who are talented. People who are capable, knowledgeable, cool and driven. Yeah. People who want their products to succeed. I never buy to destroy. Okay? I always want the new company to flourish and the software to do extremely well." What's next? "In the future, I envision that you should be able to just go around with a cam- era, shoot random things, shoot some people…I want a movie about some- thing like this. And then AI will actually make that movie for you. Not entirely. Not without supervision. Right? But it will help create something that is your vision as the creator. "I always hear this argument: AI re- placing people. Is AI creativity replacing human creativity? No, absolutely not. Actually, it's a powerful technology (that) will allow us to make better and more advanced visuals as humanity advances forward. "The company has grown. (It) started in 1995, and has grown from one person to about 70 people worldwide. It's a di©erent challenge and a di©erent kind of excitement to build the company of that size, and to serve so many customers in so many di©erent markets and so many di©erent workflows. I'm extremely excited about what I'm doing now. And like I said, 2024 is going to be a big year for everybody, and for AI." www.postmagazine.com 31 POST JAN/FEB 2024 Silhouette's new stability node can help with paint and comp work. BorisFX's Optics plug-in lets user play with lighting.

Articles in this issue

Links on this page

Archives of this issue

view archives of Post Magazine - January/February 2024