Sound for virtual worlds

© Fraunhofer IIS / David Hartfiel

Fraunhofer Cingo liefert den perfekten 3D-Sound für die Virtual Reality-Brille.

New VR devices that support Cingo® and solutions for producing 3D sound.

Two major manufacturers of consumer electronics – LG and Alcatel – added virtual reality (VR) equipment to their product ranges in 2016. The LG 360 VR headset and the Alcatel Vision both use Fraunhofer IIS Cingo® technology for playing back realistic 3D sound.

Cingo® enables users of VR devices to hear immersive 3D sound through their headphones. The playback even adapts to head movements. The right sound is crucial for creating the perfect illusion of a virtual world. If you are in a rainforest, you will hear birds rustling in the treetops, small branches snapping underfoot, and a gentle breeze blowing past. A 360° video of a rainforest on a VR headset will only be convincing if the system can achieve the correct playback of this 3D soundscape through the headphones.

But how does this sound reach the user’s ears? When producing 360° videos, the sound recording has to happen more or less invisibly. We have developed a very compact microphone array on which individual microphones can be arranged in almost any pattern. A special algorithm that works similarly to the human ear can use the recordings to produce a good reconstruction of the recording situation. This technology was used in the Wolfskinder (Wolf Children) episode of the documentary series Terra X, which is produced by German public-service broadcaster ZDF.

When it comes to postproducing the scenes in the sound studio, the aim is to create realistic 3D sounds and to skillfully place or move audio objects in the virtual world. We offer Cingo® Composer for this task. It is a plug-in designed for the most common digital audio workstations and allows sound designers and composers to mix convincing 3D sound for virtual worlds directly in their working environment of choice. Crucially, users can immediately check their work by watching the 360° video at the same time as playing back the 3D sound on headphones.

New solutions are also needed for delivering 3D sounds, and we played a major role in developing the open MPEG-H Audio standard. Its efficient coding and its support for audio channels, audio objects, and ambisonics make MPEG-H Audio ideal for delivering 3D sound in VR applications. Thanks to these Fraunhofer technologies, realistic 360° sound will finally be available in VR applications.