Publication Type
Journal Article
Version
publishedVersion
Publication Date
6-2016
Abstract
In this paper, we present the method for our submission to the emotion recognition in the wild challenge (EmotiW). The challenge is to automatically classify the emotions acted by human subjects in video clips under real-world environment. In our method, each video clip can be represented by three types of image set models (i.e. linear subspace, covariance matrix, and Gaussian distribution) respectively, which can all be viewed as points residing on some Riemannian manifolds. Then different Riemannian kernels are employed on these set models correspondingly for similarity/ distance measurement. For classification, three types of classifiers, i.e. kernel SVM, logistic regression, and partial least squares, are investigated for comparisons. Finally, an optimal fusion of classifiers learned from different kernels and different modalities (video and audio) is conducted at the decision level for further boosting the performance. We perform extensive evaluations on the EmotiW 2014 challenge data (including validation set and blind test set), and evaluate the effects of different components in our pipeline. It is observed that our method has achieved the best performance reported so far. To further evaluate the generalization ability, we also perform experiments on the EmotiW 2013 data and two well-known lab-controlled databases: CK+ and MMI. The results show that the proposed framework significantly outperforms the state-of-the-art methods.
Keywords
Emotion recognition, Video modeling, Riemannian manifold, EmotiW challenge
Discipline
Databases and Information Systems | Graphics and Human Computer Interfaces
Research Areas
Data Science and Engineering
Publication
Journal on Multimodal User Interfaces
Volume
10
Issue
2
First Page
113
Last Page
124
ISSN
1783-7677
Identifier
10.1007/s12193-015-0204-5
Publisher
Springer
Citation
LIU, Mengyi; WANG, Ruiping; LI, Shaoxin; HUANG, Zhiwu; SHAN, Shiguang; and CHEN, Xilin.
Video modeling and learning on Riemannian manifold for emotion recognition in the wild. (2016). Journal on Multimodal User Interfaces. 10, (2), 113-124.
Available at: https://ink.library.smu.edu.sg/sis_research/6404
Copyright Owner and License
Authors
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1007/s12193-015-0204-5
Included in
Databases and Information Systems Commons, Graphics and Human Computer Interfaces Commons