Delivering an end-to-end Suite for the Next Generation of Virtual Reality Audio

VR audio solutions

Please note: Starting the video transfers usage data to youtube.

The audio technologies developed by Fraunhofer IIS achieve an authentic listening experience through excellent sound quality. The institute’s Audio and Media Technologies division focuses on this objective throughout its innovations serving a broad spectrum of markets, including the rapidly emerging field of virtual reality. Fraunhofer’s audio solutions offer support in every step of the process, beginning with the creation of VR audio content, to the end result of consumers’ playback experience.

Capturing

Fraunhofer technologies for 3D audio capturing
© Photo Fraunhofer IIS

A growing number of VR cameras available today provide convincing picture quality yet the internal microphones are often unable to capture truly immersive audio. External 3D audio microphones can be expensive and require special training for optimum results, in addition to being difficult to hide from the camera’s field of view. Fraunhofer offers a straightforward solution for capturing 3D audio through a sophisticated microphone-processing algorithm that can be applied to different types of built-in or external microphones.

Production

Another important step in VR audio content production is the mixing and mastering of captured and post-produced audio. The Cingo Composer plug-in developed by Fraunhofer IIS simplifies the method of mixing 3D sound for virtual reality and supports audio objects, multichannel beds and ambisonic signals. The plug-in is compatible with major DAWs such as Nuendo, Reaper and Pro Tools. The Fraunhofer plug-in Cingo4Unity allows the authoring and monitoring of immersive audio directly into the Unity game engine before building the VR applications.

Delivery

Fraunhofer IIS soundbar prototype
© Photo Kurt Fuchs/Fraunhofer IIS

MPEG-H Audio, the next-generation audio standard for TV and VR, was developed for a listening experience going beyond surround sound. Immersive 3D sound is achieved by adding height information to the sound mix. MPEG-H bit streams can deliver channels, audio objects or scene based input (e.g. via First or Higher Order Ambisonics (FOA/HOA)). Aside from binaural rendering, MPEG-H Audio enables various speaker configurations from mono to 22.2 channels.

Learn more about MPEG-H Audio.

 

HE-AAC, a multichannel audio codec co-developed by Fraunhofer IIS, is free for VR app developers to utilize, as it is natively supported by iOS, Android, Windows and Mac OS. Content for the Hulu VR app and the LG 360 VR as well as Samsung MilkVR 360 videos are coded with HE-AAC, providing users with high quality surround sound content. The AAC codec can carry channel-based and first order ambisonics audio elements. HE-AAC is recognized world-wide for TV broadcast and streaming applications and deployed in more than eight billion devices.

Learn more about the audio codec HE-AAC.

Playback

Fraunhofer Cingo experience
© Photo David Hartfiel/Fraunhofer IIS

The post-processing technology Fraunhofer Cingo, is the solution for the delivery of immersive audio to VR devices and applications. It renders stereo, surround and immersive sound over headphones while incorporating head movements, and allows the simultaneous placement of sound objects anywhere in the virtual space around the listener. Cingo’s extensive selection of features has convinced device manufacturers such as Google, Samsung, LG and TCL Communications as well as service providers like Hulu to integrate Cingo into their products and services. For example, Cingo delivers a true VR experience to users of the Samsung Gear VR, Alcatel's VISION and the Hulu VR app.

Learn more about Fraunhofer Cingo.

Videos

Audio & Multimedia Technologies at IBC2016

We always look forward to coming to Amsterdam for the IBC exhibition: It’s the perfect place to meet up with professionals of multimedia content creation and distribution, and to present our latest Audio & Multimedia technologies.

This includes our VR audio solutions – tune in to learn more about Cingo and Co. as well as about our other exhibits for digital radio, streaming and TV broadcast, and see how they all can form the future.