Publication Type
Journal Article
Version
acceptedVersion
Publication Date
12-2022
Abstract
Converting a human portrait to anime style is a desirable but challenging problem. Existing methods fail to resolve this problem due to the large inherent gap between two domains that cannot be overcome by a simple direct mapping. For this reason, these methods struggle to preserve the appearance features in the original photo. In this paper, we discover an intermediate domain, the coser portrait (portraits of humans costuming as anime characters), that helps bridge this gap. It alleviates the learning ambiguity and loosens the mapping difficulty in a progressive manner. Specifically, we start from learning the mapping between coser and anime portraits, and present a proxy-guided domain adaptation learning scheme with three progressive adaptation stages to shift the initial model to the human portrait domain. In this way, our model can generate visually pleasant anime portraits with well-preserved appearances given the human portrait. Our model adopts a disentangled design by breaking down the translation problem into two specific subtasks of face deformation and portrait stylization. This further elevates the generation quality. Extensive experimental results show that our model can achieve visually compelling translation with better appearance preservation and perform favorably against the existing methods both qualitatively and quantitatively.
Keywords
Adaptation models, Coser portrait proxy, Deformable models, Direct mapping, Domain adaptation, Face, Portrait-to-anime translation, Shape; Simple++, Two domains
Discipline
Databases and Information Systems | Graphics and Human Computer Interfaces
Research Areas
Information Systems and Management
Publication
IEEE Transactions on Visualization and Computer Graphics
First Page
1
Last Page
17
ISSN
1077-2626
Identifier
10.1109/TVCG.2022.3228707
Publisher
Institute of Electrical and Electronics Engineers
Citation
XIAO, Wenpeng; XU, Cheng; MAI, Jiajie; XU, Xuemiao; LI, Yue; LI, Chengze; LIU, Xueting; and Shengfeng HE.
Appearance-preserved portrait-to-anime translation via proxy-guided domain adaptation. (2022). IEEE Transactions on Visualization and Computer Graphics. 1-17.
Available at: https://ink.library.smu.edu.sg/sis_research/8359
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1109/TVCG.2022.3228707
Included in
Databases and Information Systems Commons, Graphics and Human Computer Interfaces Commons