Plug-in Suite for Foundry's Nuke and Unreal

Fraunhofer IIS has developed algorithms for multi-camera content processing and implemented them in form of plugins for different software suites.

  • Realception® for Foundry´s Nuke provides several scripts and nodes that enable the creation of disparity maps from multi-camera footage. These disparity maps can be used in compositing to simplify grading or more advanced effects like virtual camera movements or synthetic defocus.
  • Realception® for Unreal Engine allows for real-time rendering of light-fields in VR experiences for a seamless composition of CGI and live-action footage.

If you would like to know more about light-field technology, please check the following link:

Realception® Plug-in suite for Foundry´s Nuke

Fraunhofer IIS provides a set of post-processing tools to work with light-field data in the professional Nuke environment for post-production.

Screenshot Nuke Cleopatra

Realception® is based on process picture data from light-field or multi-camera array recordings. The outcome is a so called disparity map – a description of the distance of two corresponding points in adjacent images. Having this profound understanding of the captured scene opens all kind of new opportunities manipulating images in a post process like the generation of depth maps.

The optimized algorithms developed by Fraunhofer offer a wide range of editing features such as:

  • Generation of depth maps and point-clouds for the creation of impressive VFX
  • Selection of virtual camera positions, orientation and focus points
  • Usage in depth-based color grading
  • Creation of 6DoF live-action content for Virtual and Mixed Reality
  • Re-lighting of objects within scenes

The plug-in suite is available for professional pilot users on request.

Realception® Tutorial "Rectification"

This video explains the rectification process based on the Realception® plug-ins. It shows how the vertical and horizontal alignment of the footage can be achieved. That forms the basis for further disparity estimation and computational imaging.

Notice: Starting the video transfers usage data to vimeo.

Realception® Plug-in for Unreal Engine

Realception® for Unreal Engine enables real-time playback of live-action video light-fields in VR worlds on different head-mounted displays. Its main application is to provide a windowed 6DoF experience with a mix of live-action footage and VR footage.

Realception® for Unreal Engine imports any light-field created by Realception® for Foundry´s Nuke. Capturing the footage in front of a green screen with a camera array permits seamless interaction between live-action footage and CGI elements providing correct perspective and stereo depth cues when watching with head-mounted displays.

Thanks to the use of light-field technology, Realception® for Unreal provides the following benefits:

  • Real-time playback of live-action videos without needing to convert the data into meshes
  • Maintenance of quality of original footage by pixel-based rendering
  • Support of complex scenes with many occlusions
  • Interaction between actors and objects across depth layers

Realception® for Unreal is available for professional pilot users on request.

More Information

Siggraph 2019, Los Angeles

Visit us at our booth 644 at the Los Angeles Convention Center at the Siggraph 2019, July 28 - August 1.


We are looking forward to seeing you!