Computer Graphics World

Edition 2 2020

Issue link: http://digital.copcomm.com/i/1277231

Contents of this Issue

Navigation

Page 64 of 67

e d i t i o n 2 , 2 0 2 0 c g w 6 3 W e increasingly have been hearing about light field technology as it pertains to simulating the "ulti- mate" visual experience. But, what exactly is light field technology, and why should we care about it? In the purest sense, a light field is "just" a mathematical function to describe light flowing in all directions through all points in a volume. When applied to cameras, where a conventional image is a captured represen- tation of light intensity, a light field image captures both the intensity and the direction of that light. When applied to display devic- es, the combined function of intensities and directions means the captured subject can be visualized from any perspective. Here, Dan Ring, head of research at Foundry, provides a primer on this technol- ogy in a Q&A with Computer Graphics World Chief Editor Karen Moltenbrey. Why is light field technology important? A conventional lens-and-sensor system collapses all of that dense light field data down into a two-dimensional image. This loses information and effectively makes final choices at capture time (about things like focus, parallax, and even camera posi- tion) that cannot later be manipulated in post-production, and also cannot be re-cre- ated for display to viewers later. But, if the richer data could be captured and displayed to the user, it would provide a viewer with a natural sense of parallax from head movements, and also allow a viewer's eye to converge and focus naturally into different depths within the scene. Deferring these photography decisions on capture allows greater freedom when visualizing. The viewing experience would then be much closer to 'being there,' in a way that stereo 3D displays have only ever approxi- mated. In addition to the viewing position and ability to focus on different depths, the added visual richness afforded to the viewer is in view-dependent surfaces. For example, metallic or speculative surfaces will appear far more 'real.' Which segments of the industry will benefit most from light field tech, and in what way? Firstly, light field tech could benefit live-ac- tion on-set capture, where the additional data could be widely exploited in post-pro- duction, particularly in visual effects where it could allow for easier or higher-quality integration of elements. Secondly, if in the future display devices were to become available at scale, post-pro- duction could pass the data through to the final display to create an unequaled viewing experience. Well before this became practical or affordable for home use, it could be a compelling part of premium cinema, in a more persistent way than stereo 3D has managed. Light field displays could also have an impact if they were used instead of LED panels in virtual production envi- ronments, as they would enable natural parallax, lighting, and focus. What about any segments outside of our industry? Light fields already have uses in industrial applications, where the extra data can be used for detailed inspection (http://raytrix. de/inspection/, for example). Visualization of products in the industrial and automotive design space will particularly benefit from light fields, where the value of more physi- cally correct light transport is important. Light field technology is not new, is it? The mathematical and physical concept of a light field dates at least back to the work of Michael Faraday in 1846 and has been continuously expanded on since then. The practical applications in photography are also not new and have been used experi- mentally for at least 100 years. It seems that suddenly we are hearing more about light field tech – why is that? Practical digital light field capture is a com- plex engineering problem as well as a com- plex computational problem. The explosion of available compute, storage, and network- ing over the last 20 years has brought the field to the cusp of being viable. Visualizing light fields had been a problem up until around 2012, when techniques for compressive sensing of light fields on the GPU were developed and used by MIT, MPI & Nvidia, for example. This enabled more content to be delivered to multi-layer display devices at higher spatial and depth resolutions, and started putting light fields under people's eyes and driving interest in the technology. How important are light fields to achiev- ing a realistic visualization experience? Simply put, if you could fully capture the complete light field of a space and fully display it back later to a viewer, it would be visually indistinguishable from the reality. In practice, of course, there will be limits on resolution and dynamic range, which make it an imperfect experience, but in principle, it still has the potential to deliver a visual experience that can't ever be achieved by two-dimensional images. A key part of a 'realistic visualization experience' is obviously the actual experi- ence. Light field display technology offers Quest for Reality A PRIMER ON LIGHT FIELD TECHNOLOGY Dan Ring

Articles in this issue

Links on this page

Archives of this issue

view archives of Computer Graphics World - Edition 2 2020