Light-field Technology

Coming Home Light-Field Movie

Notice: Starting the video transfers usage data to vimeo.

Breakdown Coming Home

Notice: Starting the video transfers usage data to vimeo.

Light-field media production system at NAB 2016

Experts from Fraunhofer IIS are presenting tools for movie production with lightfield technology. Together with the Stuttgart Media University Fraunhofer realized a project to test the requirements in a creative environemnt on a production set to learn more about the needs and to test the workflow with its light-field media production system. The results of this project as well as tools to work with light-field technology are shown at the Fraunhofer Digital Media Alliance booth SU6716.

Light-Field Data Opens up New Possibilities in Movie Production

On-location retakes are time-consuming and expensive. What if the focus was incorrectly set during shooting or the perspective has to be changed? The use of intelligent algorithms and multicamera systems that provide multiple simultaneous views opens the door to a world of new post-production possibilities.

The light-field recording system currently consists of an array of 16 high-definition cameras set up in a planar arrangement.

The material can be processed and the scenes edited as needed during post-production. The algorithms developed by Fraunhofer IIS offer a wide range of editing features such as the matrix effect, in which the actor is essentially "frozen" in the scene and the camera moves around the object. The camera perspective can be shifted or expanded, as if the camera was actually moved, even though it was actually stationary during shooting. The multiple views allow the camera position to be virtually shifted. Without actually changing the position of the cameras, the object can be moved closer to or further away from the viewer - the so-called dolly zoom effect. The system can generate HDR images from the various perspectives or create 3D views from the existing depth information. Plug-ins can be integrated into the post-production software for added creativity during the editing process.

The light-field recording system is also suitable for the production of stop-motion films where the camera remains stationary. Recent developments show the initial results of applying light field technology to moving picture content.

What is Light-Field Technology?

A light-field is defined by the number of light rays within a specific area. If all light rays are captured within a scene, this makes it possible to generate a perspective from any position, including the reconstruction of all depth of field information. Although it's not possible to capture a complete light field with current technologies, a large portion can still be acquired under special conditions. This is a major advantage when editing scenes during the subsequent postproduction process. Researchers in the Moving Picture Technologies department at Fraunhofer IIS are already demonstrating how scenes can be optimized using a limited number of perspectives and how light field technology opens up new creative opportunities.

Researchers rely on a planar camera arrangement with up to 16 HD cameras that capture slightly different scene perspectives and from which information such as the depth of field can be extracted. This makes it possible to shift the perspective and implement special effects such as virtual camera panning and zooming, all with a single light field recording.



© Photo Fraunhofer IIS/Kurt Fuchs

Lichtfeld-Aufnahmesystem und Verarbeitung von Multikameradaten

Frederik Zilly, Michael Schöberl, Peter Schäfer, Matthias Ziegler, Joachim Keinert, Siegfried Foessel, Light-field acquisition system allowing camera viewpoint and depth of field compositing in post-production.


IBC 2013, International Broadcast Convention, Amsterdam 11.-17. September 2013.