Autonomous Systems

Autonomous systems determine how a given system should behave according to many observed environmental parameters. They act independently of human supervision and choose their actions in response to the environment. Autonomous systems are comprised of sensors and components for the aggregation, analysis, and interpretation of data, as well as situation assessment, action planning, and actuators. Applications include robotic movement, controlling autonomous vehicles and drones in industrial use cases, chemical processes or hydraulic pumps in industrial plants, control of smart buildings and renewable energy production through wind turbines.

Reinforcement Learning

Popular fields in AI are unsupervised learning and supervised learning, both of which deal with static problems. Reinforcement learning is a branch of Machine Learning for dynamic optimization problems.  An agent interacts with its environment by “exploring” the data; the agent derives rules that describe how actions or decisions are carried out. The agent is guided by a reward, which can be shaped towards the desired constraints of the policy.

The behavioral policy learned by the AI-agent is therefore shaped by the training data, the state representation of the environment and the reward given by the environment. This allows to optimize the behavior towards multiple desired objectives, such as safety and effectiveness.

Dependable = Explainable and Reliable

The flexibility of designing the behavior offers the chance to tailor it towards application needs. To be trustworthy, additional measures for explainability and reliability are taken into account.  Transparency is introduced through explainable actions and understandable decisions of the AI. From the learned strategies, hierarchical rules are extracted. Reliability is ensured through explicit detection and treatment of risks in the individual application scenario. Where needed, safety can be guaranteed through formal-logic verification of extracted strategies in a formal representation of the scenario and safety constraints. Through the unified design method that makes the algorithms both explainable and reliable, a truly dependable AI is created.

Use case: Autonomous driving

As the dependable reinforcement learning is best applied in a design framework geared towards safety, we develop a dependable driving assistant in order to demonstrate our technology within the ADA Lovelace Center for Analytics, Data and Applications.

The typical processing pipeline of autonomous vehicles is made up from perception, behavioral planning, motion planning and actuators. We focus on the implementation of behavioral planning by reinforcement learning.

We show that effective driving strategies in critical scenarios can be learned in simulation. Here the strength of reinforcement learning becomes apparent: By practicing millions of hours in virtual driving school, the AI-agent can cope with critical scenarios with ease. The strategies can generalize well though domain randomization. We perform broad evaluation on widely varied scenarios. The driving behavior is optimized towards different objectives, i.e., passenger safety thorough consideration of impact zones.

Expertise

Fraunhofer IIS pools expertise essential for automated systems: from sensor fusion and machine learning to positioning and reinfocement learning. Automated offer vast potential for research and development – which we are happy to undertake in collaboration with our customers, tailored to your requirements.

Research collaboration

Together with the Friedrich-Alexander-Universität Erlangen-Nürnberg, Fraunhofer IIS is also expanding its range of teaching events, for instance for deep and reinforcement learning or machine learning.

Seminars

Fraunhofer IIS is providing online seminars in machine learning and reinforcement learning.