Multi-view face synthesis via progressive face flow
Publication Type
Journal Article
Publication Date
1-2021
Abstract
Existing GAN-based multi-view face synthesis methods rely heavily on "creating" faces, and thus they struggle in reproducing the faithful facial texture and fail to preserve identity when undergoing a large angle rotation. In this paper, we combat this problem by dividing the challenging large-angle face synthesis into a series of easy small-angle rotations, and each of them is guided by a face flow to maintain faithful facial details. In particular, we propose a Face Flow-guided Generative Adversarial Network (FFlowGAN) that is specifically trained for small-angle synthesis. The proposed network consists of two modules, a face flow module that aims to compute a dense correspondence between the input and target faces. It provides strong guidance to the second module, face synthesis module, for emphasizing salient facial texture. We apply FFlowGAN multiple times to progressively synthesize different views, and therefore facial features can be propagated to the target view from the very beginning. All these multiple executions are cascaded and trained end-to-end with a unified back-propagation, and thus we ensure each intermediate step contributes to the final result. Extensive experiments demonstrate the proposed divide-and-conquer strategy is effective, and our method outperforms the state-of-the-art on four benchmark datasets qualitatively and quantitatively.
Keywords
Faces;Face recognition;Generative adversarial networks;Image reconstruction;Facial features;Deep learning;Three-dimensional displays;Multi-view face synthesis;pose-invariant face recognition;face reconstruction
Discipline
Information Security
Research Areas
Information Systems and Management
Publication
IEEE Transactions on Image Processing
Volume
30
First Page
6024
Last Page
6035
ISSN
1057-7149
Identifier
10.1109/TIP.2021.3090658
Publisher
Institute of Electrical and Electronics Engineers
Citation
XU, Yangyang; XU, Xuemiao; JIAO, Jianbo; LI, Keke; XU, Cheng; and HE, Shengfeng.
Multi-view face synthesis via progressive face flow. (2021). IEEE Transactions on Image Processing. 30, 6024-6035.
Available at: https://ink.library.smu.edu.sg/sis_research/7875
Additional URL
https://doi.org/10.1109/TIP.2021.3090658