Personalized Driver Stress Detection with Multi-task Neural Networks using Physiological Signals

Stress can be seen as a physiological response to everyday emotional, mental and physical challenges. A long-term exposure to stressful situations can have negative health consequences, such as increased risk of cardiovascular diseases and immune system disorder. Therefore, a timely stress detection can lead to systems for better management and prevention in future circumstances. In this paper, we suggest a multi-task learning based neural network approach (with hard parameter sharing of mutual representation and task-specific layers) for personalized stress recognition using skin conductance and heart rate from wearable devices. The proposed method is tested on multi-modal physiological responses collected during real-world and simulator driving tasks.

💡 Research Summary

The paper addresses the problem of detecting driver stress in real‑time using only two non‑invasive physiological signals—heart rate (HR) and skin conductance (SC)—that can be captured by wrist‑worn wearables. Recognizing that stress responses vary widely across individuals due to factors such as age, gender, diet, and sleep, the authors adopt a multi‑task learning (MTL) framework in which each driver is treated as a separate task. The core idea is to share a global representation across all drivers while allowing a small, driver‑specific “head” that adapts the shared features to personal physiological baselines.

Two datasets are used for evaluation. The first is the public MIT Driver Stress dataset, which contains multi‑modal recordings from 17 drivers during real‑world driving (rest, city, highway). From this dataset the authors retain only hand‑wrist galvanic skin response (GSR) and heart‑rate derived from ECG, discarding EMG, respiration, foot‑GSR, and other signals because they are either noisy or impractical for everyday use. A marker channel indicating transitions between driving conditions is used to generate binary stress labels (stress vs. non‑stress). The second dataset is collected by the authors: 19 professional truck drivers performed a driving simulation while wearing wrist sensors that recorded SC at 10 Hz and HR (derived from PPG) at 1 Hz (upsampled to 10 Hz). Stress is induced with a secondary arithmetic subtraction task (the Trier Social Stress Test paradigm). Only baseline and high‑stress periods are kept for analysis.

Feature extraction follows a sliding‑window approach: 30‑second windows with 50 % overlap (step size 15 s). From each window, seven statistical descriptors are computed for HR (mean, std, min, max, range, root‑mean‑square of successive differences, and std of successive differences) and nine descriptors for SC (mean, std, min, max, range, number of peaks, peak amplitude, skewness, kurtosis). This yields a 16‑dimensional feature vector per window, which is then standardized relative to each subject’s baseline to reduce inter‑subject variability.

The neural architecture consists of a hard‑parameter‑sharing MTL network. The input layer feeds into a shared fully‑connected layer with 200 neurons and ELU activation. The output of this shared layer is branched into N driver‑specific dense layers (one per task) each containing 50 ELU‑activated neurons, followed by a sigmoid output unit. Each driver‑specific head has its own binary cross‑entropy loss; the total loss is the sum of all heads and is minimized jointly using the Adam optimizer. L2 regularization on the task‑specific layers and early stopping based on a validation set are employed to curb over‑fitting. For comparison, the authors train three baseline classifiers (logistic regression, linear SVM, RBF‑kernel SVM) and a single‑task neural network (ST‑NN) that lacks the driver‑specific heads.

Results on the MIT dataset show that the MTL network (MT‑NN) achieves an average F‑score of 0.965 ± 0.023 and Cohen’s κ of 0.879 ± 0.080, outperforming all baselines (the best baseline, SVM with RBF kernel, reaches 0.950 ± 0.027 F‑score). The ST‑NN is close (0.954 ± 0.027) but exhibits larger variance across drivers. On the simulator dataset, MT‑NN again leads with an F‑score of 0.922 ± 0.137 and κ of 0.891 ± 0.184, while the baselines suffer from high standard deviations, indicating poor generalization across subjects. The authors note that the MIT labels are derived indirectly from the marker signal and may not reflect true subjective stress, and that the simulator study’s limited duration and sample size constrain the robustness of the findings.

In the discussion, the authors argue that hard parameter sharing combined with driver‑specific heads effectively captures both common physiological patterns of stress and individual idiosyncrasies, leading to more stable performance. They acknowledge limitations: reliance on handcrafted statistical features, indirect ground‑truth labeling, and modest dataset sizes. Future work is outlined as (1) exploring transfer learning between the two datasets, (2) moving toward end‑to‑end models that ingest raw time‑series using temporal convolutions and recurrent layers, and (3) validating the models against self‑reported stress in real‑world driving scenarios.

The paper concludes that a personalized, multi‑task neural network using only HR and SC can reliably detect driver stress in both real‑world and simulated environments, provided that wearable sensors deliver clean signals. This approach paves the way for in‑vehicle stress‑aware systems that could improve driver safety and wellbeing.