Fraunhofer Institute for Integrated Circuits IIS
Additional perspectives expand possibilities in VR and video production.
One of the most promising new techniques for production of immersive media content – which gives spectators the impression of truly inhabiting a scene – such as virtual reality (VR), or composites of virtual and real images created in post-production, is light-field technology. With this method, multiple shots of a particular scene are used to create additional perspectives.
The goal is to optimize the complex and time-intensive modeling steps required for the creation of photorealistic content and subsequent rendering by using intelligent light-field algorithms. In this process, light-field technology directly captures reflections, lighting or textured surfaces. Furthermore, the different perspectives make it possible to look past objects and people and genuinely move around them in VR scenes. This is currently the most far-reaching potential application of lightfield technology: creating real content for VR glasses with practically flawless perspective.
A crucial element when working with light-field technology is a scene’s depth information and geometry, as with classical 3D reconstruction. This information is what makes it possible to correctly render masking or overlapping between individual objects in a scene when calculating new perspectives.
Our computational imaging experts are currently working on a suite of light-field software applications under the brand name Realception®. We also offer additional tools to process multi-camera content. Furthermore, in collaboration with the Max Planck Institute for Informatics, we offer a video light-field dataset for first-time users to hone the necessary skills and facilitate work with light-field material in post-production.
Our plans for further light-field developments envisage perfect 3D reconstruction of objects and scenes, complete with reflections, shadows, and much more for photorealistic content. This opens up new creative storytelling and design possibilities for computer and video games.