Emotion-sensitive robots for therapy

Robots as physical interaction partners and development tools

Some things that most people do entirely at the unconscious level pose enormous challenges for autistic children: correctly recognizing and interpreting the emotions of the person they are interacting with and responding accordingly.

The joint ERIK project therefore aims to develop a robotic platform that addresses new interaction strategies in treatments for children with impaired socioemotional functioning, such as children with autism. 

Our approach involves:

  • Integrating sensor-based emotion recognition based on facial expressions, speech and physiological signals
  • Using multiple therapeutic interaction approaches to create innovative child-robot interaction
  • Boosting autistic children’s ability to communicate through playful training and positive encouragement (e.g. for successful learning outcomes)

Real-time-capable sensors recognize pulse, facial expressions and speech for increased social interaction in daily life

Direct, real-time feedback from the robot helps children learn to correctly gauge others’ feelings, as well as become aware of their own emotions. The foundation for this is provided by both robust, video-based facial analysis methods (SHORE®) and multimodal analysis and evaluation of biosignals by means of optical pulse detection.


The planned open framework and interface design of the platform mean that other use cases are conceivable, too, such as group therapies or the treatment of dementia patients.

Opportunities and risks of human-machine interaction

The ERIK project complies with both the European General Data Protection Regulation (GDPR) and the ethical, legal and social implications (ELSI) relating to the target group (children) and incorporates these guidelines into its overall work process.

Overview of project partners and BMBF funding

The project is being funded by the German Federal Ministry of Education and Research (BMBF) (funding code 16SV7945K) for a period of three years (08/2018-08/2021). In addition to the Smart Sensing and Electronics and Supply Chain Services divisions of Fraunhofer IIS, Humboldt University of Berlin, Friedrich-Alexander-Universität Erlangen-Nürnberg, audEERING GmbH and ASTRUM IT GmbH are participating in the project.