Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

3-2023

Abstract

We propose using wrist and ear-based sensing, via multiple novel and complementary modalities, to unobtrusively infer activity-aware, complex cognitive and affective states (such as confusion, boredom, and recall failure) of individuals. While state-of-the-art wearable devices are predominantly used (a) independently, with limited coordination among multiple devices, and (b) to capture macro-level physical activity and physiological state, we seek to expand the ambit of unobtrusive wearable sensing to capture the cognitive states while performing commonplace physical activities. Such states typically manifest via fine-grained, almost unobservable, microscopic head, face, and eye movements. We identify some of these fine-grained physical markers that serve as proxies for cognitive/affective states and show that earable-mounted pressure, EMG, and ultrasonic sensing hold promise for capturing such markers.

Keywords

Earables, Multi-device sensing, Cognitive states, Acoustic sensing, Electromyography, Pressure

Discipline

Graphics and Human Computer Interfaces

Research Areas

Data Science and Engineering

Publication

Proceedings of the 2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), Atlanta, GA, March 13-17

First Page

671

Last Page

676

ISBN

9781665453813

Identifier

10.1109/PerComWorkshops56833.2023.10150270

Publisher

IEEE

City or Country

New Jersey, USA

Copyright Owner and License

Authors

Additional URL

https://doi.org/10.1109/PerComWorkshops56833.2023.10150270

Download

Share

COinS