Driver State Detection: Cognitive Load Estimation

Multimodal Dataset on psychological Overload of Drivers

Mental overload while driving, e.g. due to multitasking or diverse driver information, is a high safety risk in road traffic. The goal of modern driver monitoring systems is to monitor the driver holistically in order to be able to react to possible safety risks in real time. The detection of cognitive load is a central component for the further development of driver monitoring systems.

Therefore, we are researching robust AI-based algorithms to detect overload in real time.

Mental Overload Detection Dataset

As a basis for our AI-based algorithms, we have created a multimodal database for the detection of overstress:

  • Specific study design, precisely adapted to the requirements and environments in the vehicle.
  • Scientifically validated stimuli for the induction of overload.
  • Synchronized multimodal data acquisition for the detection of psycho-physiological states.

Study Design ADAbase: Autonomous Driving Cognitive Load Assessment Database

Stimuli for inducing overload

  • n-back test: gold standard method in psychological research
  • k-drive test: newly developed test concept, precisely adapted to the challenges of driving a car

Physiological responses:
Synchronized recordings of multimodal biosignals

  • Electroencephalography (EEG)
  • Eyetracker
  • Respiration rate
  • Thermal camera
  • Electrodermal activity (EDA)
  • Oxygen saturation (sO2)
  • Electromyography (EMG)
  • Facial expression
  • Electrocardiography (ECG)
  • Heart rate variability (HRV)
  • Photoplethysmography (PPG)
  • Electrogastrography (EGG)

Behavioral parameters

  • Face and emotion analysis: Facial Action Coding Sytstem Action Units
  • Performance parameters, such as hit rate and false positive rate
  • Reaction time
  • Direction of gaze (eye tracking)

Subjective experience measured by questionnaires

  • NASA TLX questionnaires
  • OCEAN personality traits
Affective Computing – Detection of Biosignals
© Fraunhofer IIS/

Research and Projects on AI-based Technologies for Driver Monitoring Systems.


SEMULIN – natural, multimodal interaction for automated driving

Development of a self-supporting natural human-machine interface for automated driving using multimodal input and output modes including facial expressions, gestures, gaze, and speech.


ADA Lovelace Center for Analytics, Data and Applications

New competence center for data analytics and AI that links research and industry.


Comprehensible Artificial Intelligence

In the project group Comprehensible AI, we develop methods for explainable machine learning in a cooperation of Fraunhofer IIS and the University of Bamberg.