EyeGraph : Modularity-aware spatio temporal graph clustering for continuous event-based eye tracking

Author

Panahetipola Mudiyanselage Nuwan BANDARA, Singapore Management UniversityFollow
Thivya KANDAPPU, Singapore Management UniversityFollow
Archan MISRA, Singapore Management UniversityFollow
Ila GOKARN
Archan MISRA

Publication Type

Conference Proceeding Article

Version

acceptedVersion

Publication Date

11-2024

Abstract

Continuous tracking of eye movement dynamics plays a significant role in developing a broad spectrum of human-centered applications, such as cognitive skills (visual attention and working memory) modeling, human-machine interaction, biometric user authentication, and foveated rendering. Recently neuromorphic cameras have garnered significant interest in the eye-tracking research community, owing to their sub-microsecond latency in capturing intensity changes resulting from eye movements. Nevertheless, the existing approaches for event-based eye tracking suffer from several limitations: dependence on RGB frames, label sparsity, and training on datasets collected in controlled lab environments that do not adequately reflect real-world scenarios. To address these limitations, in this paper, we propose a dynamic graph-based approach that uses a neuromorphic event stream captured by Dynamic Vision Sensors (DVS) for high-fidelity tracking of pupillary movement. More specifically, first, we present EyeGraph, a large-scale multi-modal near-eye tracking dataset collected using a wearable event camera attached to a head-mounted device from 40 participants -- the dataset was curated while mimicking in-the-wild settings, accounting for varying mobility and ambient lighting conditions. Subsequently, to address the issue of label sparsity, we adopt an unsupervised topology-aware approach as a benchmark. To be specific, (a) we first construct a dynamic graph using Gaussian Mixture Models (GMM), resulting in a uniform and detailed representation of eye morphology features, facilitating accurate modeling of pupil and iris. Then (b) apply a novel topologically guided modularity-aware graph clustering approach to precisely track the movement of the pupil and address the label sparsity in event-based eye tracking. We show that our unsupervised approach has comparable performance against the supervised approaches while consistently outperforming the conventional clustering approaches.

Keywords

Eye movement dynamics tracking, Neuromorphic cameras, Graph clustering

Discipline

Databases and Information Systems | Graphics and Human Computer Interfaces

Research Areas

Intelligent Systems and Optimization; Software and Cyber-Physical Systems

Areas of Excellence

Digital transformation

Publication

Proceedings of 38th Annual Conference on Neural Information Processing Systems (NeurIPS 2024) : Vancouver, Canada, December 10-15

Publisher

NeurIPS

City or Country

Canada

Citation

BANDARA, Panahetipola Mudiyanselage Nuwan; KANDAPPU, Thivya; MISRA, Archan; GOKARN, Ila; and MISRA, Archan. EyeGraph : Modularity-aware spatio temporal graph clustering for continuous event-based eye tracking. (2024). Proceedings of 38th Annual Conference on Neural Information Processing Systems (NeurIPS 2024) : Vancouver, Canada, December 10-15.
Available at: https://ink.library.smu.edu.sg/sis_research/9909

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.

Comments

PDF provided by faculty