Realception®

Plug-in Suite for Foundry's Nuke and Unreal

Fraunhofer IIS has developed algorithms for multi-camera content processing and implemented them in form of plugins for different software suites.

  • Realception® for Foundry´s Nuke provides several scripts and nodes that enable the creation of disparity maps from multi-camera footage. These disparity maps can be used in compositing to simplify grading or more advanced effects like virtual camera movements or synthetic defocus.
  • Realception® for Unreal Engine allows for real-time rendering of light-fields in VR experiences for a seamless composition of CGI and live-action footage.

If you would like to know more about light-field technology, please check the following link:

Realception® Plug-in suite for Foundry´s Nuke

Fraunhofer IIS provides a set of post-processing tools to work with light-field data in the professional Nuke environment for post-production.

Screenshot Nuke Cleopatra

Realception® is based on process picture data from light-field or multi-camera array recordings. The outcome is a so called disparity map – a description of the distance of two corresponding points in adjacent images. Having this profound understanding of the captured scene opens all kind of new opportunities manipulating images in a post process like the generation of depth maps.

The optimized algorithms developed by Fraunhofer offer a wide range of editing features such as:

  • Generation of depth maps and point-clouds for the creation of impressive VFX
  • Selection of virtual camera positions, orientation and focus points
  • Usage in depth-based color grading
  • Creation of 6DoF live-action content for Virtual and Mixed Reality
  • Re-lighting of objects within scenes

The plug-in suite is available for professional pilot users on request.

Realception® Tutorial "Rectification"

This video explains the rectification process based on the Realception® plug-ins. It shows how the vertical and horizontal alignment of the footage can be achieved. That forms the basis for further disparity estimation and computational imaging.

Notice: Starting the video transfers usage data to vimeo.

Realception® Plug-in for Unreal Engine

Realception® for Unreal Engine enables real-time playback of live-action video light-fields in VR worlds on different head-mounted displays. Its main application is to provide a windowed 6DoF experience with a mix of live-action footage and VR footage.

Realception® for Unreal Engine imports any light-field created by Realception® for Foundry´s Nuke. Capturing the footage in front of a green screen with a camera array permits seamless interaction between live-action footage and CGI elements providing correct perspective and stereo depth cues when watching with head-mounted displays.

Thanks to the use of light-field technology, Realception® for Unreal provides the following benefits:

  • Real-time playback of live-action videos without needing to convert the data into meshes
  • Maintenance of quality of original footage by pixel-based rendering
  • Support of complex scenes with many occlusions
  • Interaction between actors and objects across depth layers

Realception® for Unreal is available for professional pilot users on request.