Light-field Technology

What is Light-field Technology?

A light-field is defined by the light rays within a specific area. If all light rays are captured within a scene, it is possible to generate a perspective from any position. Although the complete capture of a light-field is not possible currently, a large portion can still be acquired under special conditions. This is a major advantage when editing scenes during the postproduction process.

Researchers in the Moving Picture Technologies department at Fraunhofer IIS are already demonstrating how scenes can be optimized using a set-up with professional cameras in a planar arrangement that capture slightly different scene perspectives. This opens up new creative opportunities such as virtual camera panning and zooming, all with a single light-field recording.

News

 

Realception® Plug-in Suite

Realception® is a light-field and multi camera data based Plug-in Suite for the professional movie post-production.

Download of static planar light-field data

Are you interested in light-field?
Fraunhofer IIS now offers a light-field data-set for free download.

Download of static non-planar light-field data

Are you interested in light-field?
Fraunhofer IIS now offers a light-field data-set for free download based on the paper "Non-planar inside-out dense light-field dataset and reconstruction pipeline"

New Possibilities in Movie Production

Usually, if the focus was incorrectly set during shooting or the perspective has to be changed, time-consuming and expensive retakes would be necessary. The use of intelligent algorithms and a set-up of high-quality cameras in a planar arrangement that provide multiple simultaneous views opens the door to new post-production possibilities.

The algorithms developed by Fraunhofer IIS offer a wide range of editing features, the camera perspective can, for example, be virtually shifted or expanded, as if the camera was actually moved, even though it was stationary during shooting. Another example would be the so-called dolly zoom effect, which works without moving the physical light-field camera.

The system can create 3D views from the existing depth information and plug-ins can be integrated into the post-production software for added creativity during the editing process, such as the Realception® Plug-ins for Nuke and Unreal Engine. The light-field recording system is also suitable for the production of stop-motion films where the camera remains stationary.

Projects

EU-Project RealVision - Hyperrealistic Imaging Experience

The RealVision network brings together leading universities, research centres focused on industrial development and companies to train a new generation of highly-skilled scientists and entrepreneurs in the area of hyper-realistic imaging, encoding and display technologies for the creation of high-quality accurate imaginery of realistic digital video.

The video below summarizes the team building and training school hosted by Fraunhofer IIS in the context of the Realvision project. The aim of this training school was to introduce the early stage researchers (ESRs) with topics related to light-field capturing and processing through series of lectures from the experts of the field, followed by practical sessions.

Notice: Starting the video transfers usage data to vimeo.

ETN-FPI - European Training Network on Full Parallax Imaging

ETN-FPI is a four year (2015-2019) Innovative Training Network, which has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie Grant Agreement. It brings together eight beneficiaries and eight partner organizations from Finland, Sweden, Germany, United Kingdom, Spain, Hungary and Singapore, with the aim of training a new generation of researchers in the area of full parallax imaging. This will be achieved by hiring talented 15 early stage researchers and training them to become future research leaders in this area.

Notice: Starting the video transfers usage data to vimeo.

On-line Light-field Video Processing

Fraunhofer IIS and Max-Planck Insitute for Informatics introduce a real-time multi-view correspondence algorithm that extracts multi-view depth maps from sparse, wide-baseline light-field video to produce high-quality novel views for virtual effects such as apertures or virtual camera positions.

 

"Coming Home" − First Live-Action-Clip with Light-field Processing

On set more and more cameras and camera arrays are used to generate additional views and data for visual effects or depth maps information. Together with the German Stuttgart Media University HDM, the team Computational Imaging of Fraunhofer´s Department Moving Picture Technologies set up a project to evaluate and demonstrates the potential of shooting with multi-camera arrays and light-field processing.

This project results in a short live-action pilot clip called "Coming Home" which impressively shows the possibility and performance of light-field processing in a real production workflow. The break-down movie demonstrates the requirements and conditions on set.

 

Notice: Starting the video transfers usage data to vimeo.

Breakdown Coming Home

Notice: Starting the video transfers usage data to vimeo.

Publications

"Light-field acquisition system allowing camera viewpoint and depth of field compositing in post-production"


Frederik Zilly, Michael Schöberl, Peter Schäfer, Matthias Ziegler, Joachim Keinert, Siegfried Foessel


IBC (International Broadcasting Convention), Amsterdam, Netherlands, September 13-17, 2013.

"Acquisition system for dense lightfield of large scenes"





Matthias Ziegler, Ron op het Veld, Joachim Keinert, Frederik Zilly



3DTV-Con, Copenhagen, Denmark, June 7-9, 2017.

"Immersive Virtual Reality for live-action video using camera arrays"



Matthias Ziegler, Joachim Keinert, Nina Holzer, Thorsten Wolf, Tobias Jaschke, Ron op het Veld, Faezeh Sadat Zakeri, Siegfried Foessel


IBC (International Broadcasting Convention), Amsterdam, Netherlands, September 15-19, 2017.

"A novel confidence measure for disparity maps by pixel-wise cost function analysis"

 

 

Ron Op het Veld, Tobias Jaschke, Michel   Bätz, Luca Palmieri (Department of Computer Science, Kiel University), Joachim Keinert

 

 

ICIP (International Conference on Image Processing), Athens, Greece, October 7-10, 2018.

"Benchmarking of several disparity estimation algorithms for light field processing"

 

 

Faezeh Sadat Zakeri, Michel Bätz, Tobias Jaschke, Joachim Keinert, Alexandra Chuchvara

 

 

Fourteenth International Conference on Quality Control by Artificial Vision, Mulhouse, France, May 15-17, 2019.

"Non-Planar Inside-Out Dense Light-Field Dataset and Reconstruction Pipeline"

 

 

Faezeh Sadat Zakeri, Ahmed Durmush (3D Media Group, Tampere University, Tampere, Finland), Matthias Ziegler, Michel Bätz, and Joachim Keinert

 

IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, September 22-25, 2019.

"Learning to Predict Image-based Rendering Artifacts with Respect to a Hidden Reference Image"

 

 

Mojtaba Bemana (MPI Informatik), Joachim Keinert, Karol Myszkowski, Michel Bätz, Matthias Ziegler, Hans-Peter Seidel (MPI Informatik), Tobias Ritschel (University College London)

 

 

 

More Information

Download of static planar light-field data

Are you interested in light-field?
Fraunhofer IIS now offers a light-field data-set for free download.

Back to Research in the field of Moving Picture Technologies

Download of static non-planar light-field data

Are you interested in light-field?
Fraunhofer IIS now offers a light-field data-set for free download based on the paper "Non-planar inside-out dense light-field dataset and reconstruction pipeline"