Center for Digital Signal Processing using Artificial Intelligence DSAI

Fraunhofer IIS combines digital signal processing with artificial intelligence in a center for research, consulting and licensing and for creating special customized solutions.

Today’s consumer electronics, automotive technology, telecommunications and intelligent assistance systems would be inconceivable without digital signal processing. Now we are taking the logical next step by using machine learning and artificial intelligence to advance these fields of application even further. This will allow us to find solutions for problems in our core competence domains that would not be obtainable through traditional digital signal processing. In view of the general significance of signal processing and AI across various applications, it is essential for companies to be able to access excellent basic technologies for their products.

To ensure that they can, Fraunhofer IIS is building up its Center for Digital Signal Processing using Artificial Intelligence DSAI, bringing together all relevant knowledge under one roof.

Core areas

The DSAI addresses the three economically significant topics that are also the focus of Fraunhofer IIS’s core competencies: computer vision, voice signal processing and signal processing for data transmission. Each of these areas presents opportunities for using AI to improve on the results achieved to date.

Computer Vision

Privacy warning

With the click on the play button an external video from www.youtube.com is loaded and started. Your data is possible transferred and stored to third party. Do not start the video if you disagree. Find more about the youtube privacy statement under the following link: https://policies.google.com/privacy

A comparison of traditional automatic 3D reconstruction and image synthesis versus AI-based automatic 3D reconstruction and image synthesis. The improvements are most easily visible in homogeneous areas (i.e. display of the oscilloscope, side of the brown bucket), as well as for objects with complex surfaces (i.e. blue flower vase) or fine details (cable connections of the oscilloscope).

Understanding geometry and the material properties of objects and scenes is a key technology in film production. It also opens the door to new possibilities in three-dimensional audio playback, immersive communication, mobility and automation and logistics.

The DSAI is striving to greatly enhance this key technology by combining previous research findings with the new possibilities offered by machine learning. Its central task is to tap the potential of significantly better 3D reconstructions and make this available for practical applications.

In the long term, this should make it possible to get a much more realistic impression of the visualized environment than has been gained with a traditional (two-dimensional) image. One potential future application is virtual real estate viewings: realtors can use their smartphone to photograph the interior of the property by moving the device slowly through the space – similar to how they would produce a panoramic image. Thanks to AI methods that make it easier to produce enhanced digitalized environments, the result is a 3D visualization. This provides potential buyers with the feeling of actually being in the space. If the viewing is to take place online, it is also necessary to find an efficient solution for transmitting the data. To this end, the DSAI project will build on our preliminary work on image compression to develop solutions based on machine learning for transmitting 3D image information more efficiently.

Audio signal processing

© monsitj - stock.adobe.com

For over 30 years, the Audio and Media Technologies division has been shaping the standards and technologies used worldwide in the audio and film industries. Today, Fraunhofer IIS systems and tools for the creation, transmission and provision of outstanding audio and video content are used in several sectors, making it possible to have real-time communication with top-quality audio. Audio and media technologies made in Erlangen, Germany, can be found in almost all computers, cell phones and consumer electronics and are used every day by billions of people all over the world. In the realm of audio signal processing, Symphoria and the Sonamic product family provide an immersive and optimized sound experience in cars, while the upHear product family dramatically enhances 3D audio playback and recording quality for professional and consumer devices. In the future, AI algorithms will provide the keys to overcome problems in audio signal processing that conventional methods have consistently failed to solve. For instance, it is possible to train AI to recognize and isolate spoken dialogue in existing film material, thus allowing users to make what is spoken louder than the background noise of the scene. Now that the first successful tests involving public service broadcasters have been performed, this technology is to be expanded further. When it comes to playback, combining smart speakers featuring built-in microphones with AI-based methods provides information on the geometry of the room, which can then be used to automatically calibrate playback for that specific environment.

Natural Language User Interface (NLUI)

© Fraunhofer IIS / hl-studios

Even though many of today’s applications can be controlled using voice commands, the world has only just begun to explore what the technology can do. Traditional signal processing methods aren’t able to handle dynamic and noisy acoustic environments. Moreover, conventional speech synthesis processes are difficult to set up and lack a certain naturalness. Modeling the vast number of parameters that influence playback quality is hard to do using conventional methods. This is where we want to apply AI-based methods to help enhance voice-controlled human-machine interaction. For instance, this would allow voice assistants to perform better in acoustically demanding environments and to communicate information efficiently using clear, understandable natural language. Fraunhofer IIS and IAIS are collaborating on speech assistance technologies under the brand of Allinga, offering all modules of a voice assistant from in-house development. Next to being easy adaptable to domain specific use cases, Allinga is committed to data sovereignty on all levels.

In the Speaker projectAllinga technology is implemented in a B2B voice assistance platform, enabling renowned industry partners of the project to realize use-cases that can be readily transferred into practice.

 

Data transmission

© your123 - stock.adobe.com

Fraunhofer IIS has more than 20 years of experience in developing optimized wireless transmission systems. Regardless of whether transmission is via satellite, a mobile network or in the Internet of Things, our researchers are working on technologies that can be tailored to any application. Fraunhofer IIS energy-saving embedded AI algorithms pave the way for real cognitive sensors.

The DSAI combines AI know-how with expertise in communication technologies to harness AI-based signal processing to meet the challenges that future communication networks will face. Distributed learning, distributed inference and fog computing are key components in next-generation IoT sensor networks and mobile networks, in which algorithms are distributed across multiple nodes. Joint source and channel coding provide for greater energy and bandwidth efficiency. Meanwhile, AI-based algorithms for sharing the limited frequency spectrum guarantee optimized use of communication channels. Using spiking neural networks (SNNs) paves the way for efficient signal processing and transmission. Given the trend toward using higher transmission frequencies for 5G and 6G, interest in shared wireless communication and radar-sensing systems is growing. AI is expected to help optimize resource allocation in this area. A universal demonstration platform for the use of AI algorithms in communication systems rounds off the research work.

Computer Vision: Photogrammetry and Neural Radiance Fields - Chances and Challenges

Join us!

Have you always been fascinated by AI? Are you interested in pursuing your ideas and scientific curiosity about AI-based signal processing? Then why not apply for a position with us? Using “Speculative application DSAI” as the subject line, send your application to personalmarketing@iis.fraunhofer.de.

Logo StMWi
© StMWi

The DSAI Center has been awarded funding by the Bavarian Ministry of Economic Affairs, Regional Development and Energy in the amount of 13 million euros for the period 2020 to 2025.

Back to Audio and Media Technologies

Back to Communication Systems