Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

3-2023

Abstract

We propose using wrist and ear-based sensing, via multiple novel and complementary modalities, to unobtrusively infer activity-aware, complex cognitive and affective states (such as confusion, boredom, and recall failure) of individuals. While state-of-the-art wearable devices are predominantly used (a) independently, with limited coordination among multiple devices, and (b) to capture macro-level physical activity and physiological state, we seek to expand the ambit of unobtrusive wearable sensing to capture the cognitive states while performing commonplace physical activities. Such states typically manifest via fine-grained, almost unobservable, microscopic head, face, and eye movements. We identify some of these fine-grained physical markers that serve as proxies for cognitive/affective states and show that earable-mounted pressure, EMG, and ultrasonic sensing hold promise for canturing such markers.

Keywords

Earables, Multi-device sensing, Cognitive states, Acoustic sensing, Electromyography, Pressure

Discipline

Data Science | Graphics and Human Computer Interfaces

Research Areas

Data Science and Engineering

Publication

Proceedings of the Ninth Workshop on sensing systems and applications using wrist worn smart devices, Atlanta, Georgia, 2023 March 13-17

First Page

671

Last Page

676

ISBN

9781665453813

Identifier

10.1109/PerComWorkshops56833.2023.10150270

Publisher

IEEE Computer Society

City or Country

Atlanta, Georgia

Embargo Period

6-25-2023

Copyright Owner and License

Authors

Additional URL

https://doi.org/10.1109/PerComWorkshops56833.2023.10150270

Download

Share

COinS