Publication Type

Journal Article

Version

publishedVersion

Publication Date

1-2018

Abstract

The paper explores the possibility of using wrist-worn devices (specifically, a smartwatch) to accurately track the hand movement and gestures for a new class of immersive, interactive gesture-driven applications. These interactive applications need two special features: (a) the ability to identify gestures from a continuous stream of sensor data early–i.e., even before the gesture is complete, and (b) the ability to precisely track the hand’s trajectory, even though the underlying inertial sensor data is noisy. We develop a new approach that tackles these requirements by first building a HMM-based gesture recognition framework that does not need an explicit segmentation step, and then using a per-gesture trajectory tracking solution that tracks the hand movement only during these predefined gestures. Using an elaborate setup that allows us to realistically study the table-tennis related hand movements of users, we show that our approach works: (a) it can achieve 95% stroke recognition accuracy. Within 50% of gesture, it can achieve a recall value of 92% for 10 novice users and 93% for 15 experienced users from a continuous sensor stream; (b) it can track hand movement during such stroke play with a median accuracy of 6.2 cm

Keywords

VR, gesture recognition, hand tracking, immersive applications, low-latency, wearable devices

Discipline

Information Security | Software Engineering

Research Areas

Software and Cyber-Physical Systems

Publication

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Volume

2

Issue

1

First Page

39: 1

Last Page

27

ISSN

2474-9567

Identifier

10.1145/3191771

Publisher

Association for Computing Machinery (ACM)

Copyright Owner and License

Authors

Additional URL

https://doi.org/10.1145/3191771

Download

Share

COinS