Publication Type

Journal Article

Version

publishedVersion

Publication Date

9-2021

Abstract

An empathetic car that is capable of reading the driver’s emotions has been envisioned by many car manufacturers. Emotion inference enables in-vehicle applications to improve driver comfort, well-being, and safety. Available emotion inference approaches use physiological, facial, and speech-related data to infer emotions during driving trips. However, existing solutions have two major limitations: Relying on sensors that are not built into the vehicle restricts emotion inference to those people leveraging corresponding devices (e.g., smartwatches). Relying on modalities such as facial expressions and speech raises privacy concerns. By contrast, researchers in mobile health have been able to infer affective states (e.g., emotions) based on behavioral and contextual patterns decoded in available sensor streams, e.g., obtained by smartphones. We transfer this rationale to an in-vehicle setting by analyzing the feasibility of inferring driver emotions by passively interpreting the data streams of the control area network (CAN-bus) and the traffic context (inferred from the front-view camera). Therefore, our approach does not rely on particularly privacy-sensitive data streams such as the driver facial video or driver speech, but is built based on existing CAN-bus data and traffic information, which is available in current high-end or future vehicles. To assess our approach, we conducted a four-month field study on public roads covering a variety of uncontrolled daily driving activities. Hence, our results were generated beyond the confines of a laboratory environment. Ultimately, our proposed approach can accurately recognise drivers’ emotions and achieve comparable performance as the medical-grade physiological sensor-based state-of-the-art baseline method.

Keywords

Emotion recognition, Driving behaviours, Traffic contexts, Control area network (CAN), Intelligent vehicle

Discipline

Operations Research, Systems Engineering and Industrial Engineering | Software Engineering

Research Areas

Software and Cyber-Physical Systems

Publication

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Volume

5

Issue

3

First Page

1

Last Page

34

ISSN

2474-9567

Identifier

10.1145/3478078

Publisher

ACM

Copyright Owner and License

Publisher

Additional URL

https://doi.org/10.1145/3478078

Download

Share

COinS