Multimodal Datasets for Driver Monitoring

Multimodal Dataset on psychological Overload of Drivers

Mental overload while driving, e.g. due to multitasking or diverse driver information, is a high safety risk in road traffic. The goal of modern driver monitoring systems is to monitor the driver holistically in order to be able to react to possible safety risks in real time. The detection of cognitive load is a central component for the further development of driver monitoring systems.

Therefore, we are researching robust AI-based algorithms to detect overload in real time.

Mental Overload Detection Dataset

As a basis for our AI-based algorithms, we have created a multimodal database for the detection of overstress:

Specific study design, precisely adapted to the requirements and environments in the vehicle.
Scientifically validated stimuli for the induction of overload.
Synchronized multimodal data acquisition for the detection of psycho-physiological states.

Study Design ADAbase: Autonomous Driving Cognitive Load Assessment Database

Stimuli for inducing overload

n-back test: gold standard method in psychological research
k-drive test: newly developed test concept, precisely adapted to the challenges of driving a car

Physiological responses:
Synchronized recordings of multimodal biosignals

Electroencephalography (EEG)
Eyetracker
Respiration rate
Thermal camera
Electrodermal activity (EDA)
Oxygen saturation (sO2)
Electromyography (EMG)
Facial expression
Electrocardiography (ECG)
Heart rate variability (HRV)
Photoplethysmography (PPG)
Electrogastrography (EGG)

Behavioral parameters

Face and emotion analysis: Facial Action Coding Sytstem Action Units
Performance parameters, such as hit rate and false positive rate
Reaction time
Direction of gaze (eye tracking)

Subjective experience measured by questionnaires

NASA TLX questionnaires
OCEAN personality traits

Affective Computing – Detection of Biosignals — © Fraunhofer IIS/iStock.com-angelhell

Research and Projects on AI-based Technologies for Driver Monitoring Systems.

SEMULIN – natural, multimodal interaction for automated driving

Development of a self-supporting natural human-machine interface for automated driving using multimodal input and output modes including facial expressions, gestures, gaze, and speech.

project SEMULIN – more information

ADA Lovelace Center for Analytics, Data and Applications

New competence center for data analytics and AI that links research and industry.

Zum ADA Lovelace Center

Comprehensible Artificial Intelligence

In the project group Comprehensible AI, we develop methods for explainable machine learning in a cooperation of Fraunhofer IIS and the University of Bamberg.

Transparency and Intelligibility of AI Systems