Deep learning of facial embeddings and facial landmark points for the detection of academic emotions
Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
7-2020
Abstract
Automatic emotion recognition is an actively researched area as emotion plays a pivotal role in effective human communications. Equipping a computer to understand and respond to human emotions has potential applications in many fields including education, medicine, transport and hospitality. In a classroom or online learning context, the basic emotions do not occur frequently and do not influence the learning process itself. The academic emotions such as engagement, frustration, confusion and boredom are the ones which are pivotal to sustaining the motivation of learners. In this study, we evaluated the use of deep learning on FaceNet embeddings and facial landmark points for academic emotion detection on a publicly available dataset - DAiSEE that has been annotated with the emotional states of engagement, boredom, frustration and confusion. By modeling both the spatial and temporal dimensions, the results demonstrated that both models are able to detect incidences of boredom and frustration and can be used in the moment-by-moment monitoring of boredom and frustration of learners using a tutoring system either online or in a classroom.
Keywords
datasets, deep learning, emotions, facial emotion recognition
Discipline
Databases and Information Systems | Educational Assessment, Evaluation, and Research | Graphics and Human Computer Interfaces | Higher Education
Research Areas
Information Systems and Management
Publication
ICIEI 2020: Proceedings of the 5th International Conference on Information and Education Innovations, July 26-28, London
First Page
111
Last Page
116
ISBN
9781450375757
Identifier
10.1145/3411681.3411684
Publisher
ACM
City or Country
New York
Citation
FWA, Hua Leong.
Deep learning of facial embeddings and facial landmark points for the detection of academic emotions. (2020). ICIEI 2020: Proceedings of the 5th International Conference on Information and Education Innovations, July 26-28, London. 111-116.
Available at: https://ink.library.smu.edu.sg/sis_research/6859
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1145/3411681.3411684
Included in
Databases and Information Systems Commons, Educational Assessment, Evaluation, and Research Commons, Graphics and Human Computer Interfaces Commons, Higher Education Commons