Fraunhofer upHear® Spatial Audio Microphone Processing

Overview

About

The Fraunhofer spatial audio capturing algorithm has been designed to significantly improve the sound capture capabilities of professional and consumer 360° cameras and mobile devices using built-in microphones.

It is the first audio technology to be delivered under Fraunhofer’s upHear brand of immersive audio innovations.

The algorithm automatically transforms the captured sound in real-time to any popular surround or immersive audio reproduction format, such as FOA, HOA, 5.1 channels, and 7.1+4 height channels, while preserving the authenticity of the audio scene.

 

Challenge

While the availability and performance of 360° cameras are improving, the devices often lack support for capturing sounds that represent the entire space. Instead, the users are often left with mono or stereo output which is inappropriate for immersive experiences.

Capturing spatial audio is traditionally achieved with complicated microphone arrays or expensive ambisonics microphones. Both require the specialized knowledge of a sound engineer for operating and also ensuring that the microphones do not obstruct the camera’s field-of-view. Moreover, most external microphone solutions cannot accommodate the needs of complex recording situations where the camera needs to be placed in tight spaces, or quickly moved from one scene to another.

Until now, these problems have been making spatial audio capturing a time-consuming, error-prone task that is usually left to professionals or audio enthusiasts to carry out.

 

Benefits

Fraunhofer’s spatial audio microphone processing technology enables anyone capable of operating a camera to enhance video content with spatial audio in a simple and robust way.

Professional Content Creators

Professional content creators can also benefit from Fraunhofer’s audio expertise which helps them capture high-quality, position-accurate 3D audio and a natural ambience that carries the atmosphere of the set. In post-production, the captured audio scenes can be combined with additional sound elements such as voice overs or dynamic objects.

Consumers

Consumers are usually looking for products that are easy to use and can provide high-quality recordings. Fraunhofer’s spatial audio capturing technology integrated in consumer cameras enables exactly that: immersive sound capture to accommodate the video with the push of a button.

Product Features

For the manufacturer

  • Spatial audio capturing optimized for unique camera or mobile designs
  • Runs on the device, in an app, or in post-production software
  • Resource-optimized for mobile platforms

For the consumer

  • Exported spatial audio format is ready for upload to YouTube and Facebook
  • Real-time audio output, suitable for live-streaming directly from the device
  • Captures directional sounds and enveloping ambiences accurately

Microphone Requirements

Fraunhofer Spatial Audio Microphone Processing adapts to the unique body and microphone configuration of the device. This enables flexibility in the product design, and ensures optimal audio recording.

The number of microphones needed on the device can typically range from 3 (consumer products) to 8 (professional devices). Configurations shown in the following graphic are only examples.

 

Availability

The spatial audio capturing algorithm can be provided as an optimized library for:

  • Desktop Software (Windows/Mac)
  • Mobile Apps (iOS/Android)
  • Embedded Systems (e.g., ARM Cortex)

It is available for licensing to 360° camera and mobile phone manufacturers.

Interested in upHear®?

For a consultation regarding your device’s microphone placement and spatial audio capability: