DR-FER: Discriminative and Robust Representation Learning for Facial Expression Recognition

Publication Type

Journal Article

Publication Date

12-2023

Abstract

Learning discriminative and robust representations is important for facial expression recognition (FER) due to subtly different emotional faces and their subjective annotations. Previous works usually address one representation solely because these two goals seem to be contradictory for optimization. Their performances inevitably suffer from challenges from the other representation. In this paper, by considering this problem from two novel perspectives, we demonstrate that discriminative and robust representations can be learned in a unified approach, i.e. DR-FER, and mutually benefit each other. Moreover, we make it with the supervision from only original annotations. Specifically, to learn discriminative representations, we propose performing masked image modeling (MIM) as an auxiliary task to force our network to discover expression-related facial areas. This is the first attempt to employ MIM to explore discriminative patterns in a self-supervised manner. To extract robust representations, we present a category-aware self-paced learning schedule to mine high-quality annotated ( easy ) expressions and incorrectly annotated ( hard ) counterparts. We further introduce a retrieval similarity-based relabeling strategy to correct hard expression annotations, exploiting them more effectively. By enhancing the discrimination ability of the FER classifier as a bridge, these two learning goals significantly strengthen each other. Extensive experiments on several popular benchmarks demonstrate the superior performance of our DR-FER. Moreover, thorough visualizations and extra experiments on manually annotation-corrupted datasets show that our approach successfully accomplishes learning both discriminative and robust representations simultaneously.

Keywords

Annotations, Artificial neural networks, Electronic mail, Facial expression recognition, masked image modeling, Representation learning, Schedules, self-paced learning, Task analysis, Training

Discipline

Databases and Information Systems | Graphics and Human Computer Interfaces

Research Areas

Software and Cyber-Physical Systems

Publication

IEEE Transactions on Multimedia

First Page

1

Last Page

14

ISSN

1520-9210

Identifier

10.1109/TMM.2023.3347849

Publisher

Institute of Electrical and Electronics Engineers

Additional URL

https://doi.org/10.1109/TMM.2023.3347849

This document is currently not available here.

Share

COinS