Fraunhofer’s New Interactive 3D Audio System

HEAR YOUR HOME TEAM™

Fraunhofer’s New Interactive And Immersive Audio System for TV Broadcasting

Based on the MPEG-H Standard and A/342 Candidate Standard for ATSC 3.0

Overview

Fraunhofer IIS MPEG-H Interaktivität
© Photo Fraunhofer IIS/Frank Boxler/Valentin Schilling

Meeting the needs of the 21st century viewer

Television viewers today consume content on a variety of devices and in many cases their interest in TV sound has begun to focus more on unobtrusive convenience than on reproduction quality. These consumers are also used to the more interactive and personalized experiences offered by new media today.
How may broadcasters retain their viewing audiences in the future? One answer may be to offer a compelling audio experience on all the target devices consumers use, while allowing them to tailor that experience to their liking.  
Fraunhofer’s new TV audio system, now under development, and being considered for upcoming standards such as ATSC 3.0, will offer broadcasters the ability to do this.

 

Hear your home team™: Interactivity offers a personalized listening experience

Our system will offer interactivity using MPEG-H’s object coding,  which allows viewers to adjust the sound mix to their preferences, boosting hard-to-understand dialogue or creating a “home team“-mix of sports broadcasts. This feature may also be used to efficiently add objects for dialogue in additional languages or VI descriptions to a broadcast, spending only 20-40 kb/s for each language. Although it is based on different technology, this interactivity builds upon Fraunhofer’s pioneering trials with the BBC on the Dialogue Enhancement system dubbed the Wimbledon “Grunt Controller”.

 

Immersive sound offers cinema-like realism

The system may transmit immersive sound with additional front and rear height speaker channels or the new Higher-Order Ambisonics sound field technology, improving today’s surround sound broadcasts to provide a truly realistic and immersive audio experience on par with the latest cinema sound systems.
Cinemas are starting to be equipped for 3D or immersive sound and feature films released with 3D mixes.  Sound reproduction in a theater offers different challenges and is not a planned application for our system, but it is possible to ingest and transmit a feature film with 3D sound to the home. More importantly, 3D sound offers broadcasters the opportunity to create a more realistic audio experience for their own content.
As shown in the figure below, double-blind studies at Fraunhofer show a substantial improvement in overall sound quality when four extra channels are added as signals for height speakers.

Overall sound quality improvement with expansion of reproduction system to surround and immersive/3D formats compared to 22.2 channel reference signal.

 

Immersive sound may also be enjoyed in the future with devices such as Fraunhofer’s conceptual prototype of a “3D sound bar”, enabling mainstream consumers to experience high-quality immersive audio without the complexity of adding new speakers.

 

Unsurpassed coding efficiency based on AAC evolution plus new technologies

Since the system uses a new audio codec based on the evolution of the AAC family as well as new technologies, the additional channels or objects needed for 3D sound can be transmitted at bitrates similar to those used today for 5.1 surround broadcasts.

Bit rates in kbit/s for

Good

Recommended

Transparent

22.2 Channels

256

512

1200

7.1 + 4 Height Channels

+ 4 Objects

200

384

800

5.1 Channels

96

160

256

2.0 Channels

32

56

160

 

Internet-ready for a great listening experience on every device

Building on our experience developing HE-AAC, the native surround audio codec of iOS and Android, our system will also offer DASH support for stutter-free streaming and audio I-frames for easy DASH bitstream switching and easy splicing for ad insertion. It includes multi-platform loudness control to provide a tailored experience for a viewer’s device and listening environment.

 

Speaker-foolproof rendering for the best sound from legacy speakers

Improved rendering technologies in the system offer the ability to play any content format on any speaker configuration and may be able to correct for misplaced speakers in the consumer’s listening room The renderer also offers improved downmix quality by avoiding signal cancellation and may render a limited impression of height without height speakers, for legacy 5.1 or 2.0 consumers.

 

Easy, staged implementation for broadcast facilities and Networks

Our system is backward-compatible with the systems and practices used today for AC-3 or HE-AAC surround sound broadcasting and offers a staged approach to implementing new features. We envision four stages:

  1. Transmission of today’s surround sound, with current metadata , loudness profiles, and practices – but with ½ the bitrate due to improved coding efficiency
  2. Addition of interactive objects
  3. Addition of 3D sound with height channels or Higher-Order Ambisonics
  4. Addition of dynamic objects (audio objects panned to track video action)

Decoders in our system will offer support for all four stages from the beginning, and broadcasters may use their features when their facilities and content support them.

 

The AAC family: 15 Years of proven service, providing half the world’s TV surround sound

Fraunhofer’s new TV audio system builds upon fifteen years of experience providing surround sound using the AAC codec family around the world. Fraunhofer designed the first AAC encoding hardware for Japanese HDTV in 1998 and AAC 5.1 decoding is supported in all Japanese TVs, AVRs, and soundbars. AAC is also the current codec of NHK’s new 22.2 channel audio system for Super Hi-Vision broadcasting.

This work in Japan lead to the adoption of HE-AAC Multichannel as the surround codec of the ISDB-based SBTVD system used in many South American countries, and today Brazilian broadcasters, including TV Globo, use HE-AAC to provide 5.1 surround sound to their viewers.

HE-AAC is also used for surround sound in many countries using the DVB system and the AAC family has been on the air since 2009 providing surround sound for the BBC’s HD service in the UK.

 

The MPEG-H Audio Alliance: Bringing MPEG-H to the World’s Devices

Two industry leaders are collaborating to make open standards-based MPEG-H Audio a reality.  Each one contributes key technical elements as well as long-standing expertise within their market segments.

  • Fraunhofer IIS is the principal developer of MPEG audio standards, providing half the world’s TV surround sound, with its technology already used in more than 7 billion devices today. 
  • Qualcomm Technologies is the primary global supplier of chipsets and software for mobile devices delivered around the globe.

The combined experience and broad-market presence represented by these three companies has not only been critical in developing MPEG-H, but also provides the foundation for successfully bringing exciting new audio features to market.

Papers

 

Technical Brief: Installing the MPEG-H Audio Alliance’s New Interactive and Immersive TV Audio System in Professional Production and Distribution Facilities

Robert Bleidt - 2015

 

  • Read paper: Installing MPEG-H Audio Alliance's New Interactive and Immersive TV Audio System in Professional Production and Distribution Facilities

 

 

Immersive Audio with MPEG 3D Audio - Status and Outlook

Robert Bleidt, Stefan Meltzer, Max Neuendorf, Jan Plogsties - NAB 2014 (Printed in the NAB Broadcast Engineering Conference Proceedings 2014, presented at the 2014 NAB Show.)

 

 

 

Object-Based Audio: Opportunities for Improved Listening Experience and Increased Listener Involvement

Robert Bleidt, Arne Borsum, Harald Fuchs, S. Merrill Weiss - SMPTE 2014 (Written for SMPTE Journal©)

 

News

Korean Manufacturers License Fraunhofer’s MPEG-H Audio Technology For Broadcast Encoders

Now that Korea has adopted the MPEG-H Audio standard for its nationwide advanced (“immersive”) audio broadcasts, several encoder manufacturers have begun to adopt specific software algorithms associated with the technology from Fraunhofer IIS, based in Germany.

AES 2015 Reflections: Fraunhofer USA’s Robert Bleidt discusses the ‘four phases’ of immersive audio adoption

For German research organisation Fraunhofer, it is the MPEG-H standard that is leading the way to a more immersive audio future. As part of the MPEG-H Audio Alliance alongside Qualcomm Technologies and Technicolor, Fraunhofer is helping to spread the word about a system that will deliver comprehensive object-based audio.

Choice of ATSC 3.0 Audio System Comes Down to the Wire

The choice between systems for ATSC 3.0 audio is now down to MPEG-H versus Dolby AC-4 and there is about equal support for each system. It’s now mired in politics within the committee, though a decision is due by the end of 2015.

Considerations for the MPEG-H Audio Standard

This article discusses the plan for the new MPEG-H Audio standard, a real-time 3-D audio encoder system with applications for broadcast and streaming distribution, developed in large part by the Fraunhofer Institute in Germany, and recently finalized and submitted as an ISO/MPEG standard.

Demos of ATSC 3.0 Immersive-Audio Format Offer Chance To Grow Alliances

This week’s special event organized by the ATSC S34-2 working group offered real-time comparisons of candidates for the audio aspect of the ATSC 3.0 standard. Although no conclusions are scheduled to be announced until the fall, the event, held in Atlanta, has become an opportunity for third-party broadcast-technology–equipment manufacturers to line up behind either Dolby’s AC-4 or the MPEG-H Audio Alliance of Fraunhofer, Technicolor, and Qualcomm.

MPEG-H Audio Brings New Features to TV and Streaming Sound

New TV audio systems such as MPEG-H not only offer consumers more accurate sound, but the ability to adjust the sound to their preferences and hear it as it can best be reproduced on their devices.

Implementing MPEG-H Audio in Television Broadcast Operations

The coding efficiency of audio compression is about to improve markedly because of a new MPEG audio standard—MPEG-H. This standard will permit many new and useful capabilities for those involved in television audio.

The Sound of Evolution

Winner of five television Emmys, Dennis Baxter, a broadcast sound designer, author, columnist, lecturer and critic within the sports broadcast industry gives his viewpoint on the next generation of audio technologies.

Fraunhofer IIS Shows Virtues of MPEG-H for Enhanced Audio in ATSC 3.0

MPEG-H is a new about-to-be adopted standard that offers object oriented audio for TVs and mobile devices. It was demonstrated at the SMPTE Fall conference and is vying to be included in the next generation broadcast standard – ATSC 3.0 – Chris Chinnock

MPEG-H Audio @ IBC 2014

Fraunhofer demo rated as one of the best ideas at IBC 2014 by nScreenNoise.

Videos

Please note: Starting the videos transfers usage data to youtube.

MPEG-H Audio for Next-Generation TV Broadcasting at IBC 2015

MPEG-H Audio Alliance Live Broadcast Demonstration at NAB 2015

Jünger Audio, Fraunhofer & MPEG-H

 

MPEG-H Enables Interactive Audio, Immersive Sound, and Maximizes Sound on All Devices

Electronic Design Technology Editor Bill Wong talks with Robert Bleidt, Division General Manager for Fraunhofer USA Digital Media Technologies, about the new MPEG-H Audio standard.