Process steps towards automated action
The key point is that technical “assistants” are playing an increasingly important role in everyday life and their functions and responsibilities are becoming more and more complex. As a general rule, environmental data collected by sensors must be processed and evaluated. Take the flood of data provided by smartphones; in the near future, this is to be individually filtered instead of having references and notifications add to it. But user experience is improved only when this selection is automated. The next step will be when this kind of machine learning becomes established in everyday life, which means we will all encounter it in a variety of ways. However, the basic principle is the same: systems learn to provide tailored information and supply only the data necessary. Fraunhofer IIS puts its main focus on local, real-time systems without access to vast amounts of data or computing capacity.
The institute has the requisite expertise in sensor fusion, machine learning, positioning and SLAM. What data gets collected and evaluated is part of the automated action that depends on the individual requirements placed on the system. So the “assistant” evaluates a situation and generates options for action; the actions themselves are divided into long-term/high-level and short-term/low-level.
In the past few years, one thing that has become particularly important is reinforcement learning, whereby an agent interacts with its environment to independently draft a strategy for solving a given problem. To do so, the agent independently “explores” the requisite data set in order to derive rules for how actions or decisions are carried out. This eliminates the need for supervised learning from gathered data.
Applications for reinforcement learning include the control of machines or vehicles, since these are areas in which data is not available in advance. In traditional control engineering, (physical) models are used to design certain rules. But here too, each model represents a mere approximation of reality. The goal of reinforcement learning is then either to approximate the model and integrate it into a set of traditional rules or to directly replace the rules as end-to-end learning.