Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

10-2021

Abstract

Existing GAN inversion methods are stuck in a paradox that the inverted codes can either achieve high-fidelity reconstruction, or retain the editing capability. Having only one of them clearly cannot realize real image editing. In this paper, we resolve this paradox by introducing consecutive images (e.g., video frames or the same person with different poses) into the inversion process. The rationale behind our solution is that the continuity of consecutive images leads to inherent editable directions. This inborn property is used for two unique purposes: 1) regularizing the joint inversion process, such that each of the inverted codes is semantically accessible from one of the other and fastened in an editable domain; 2) enforcing inter-image coherence, such that the fidelity of each inverted code can be maximized with the complement of other images. Extensive experiments demonstrate that our alternative significantly outperforms state-of-the-art methods in terms of reconstruction fidelity and editability on both the real image dataset and synthesis dataset. Furthermore, our method provides the first support of video-based GAN inversion and an interesting application of unsupervised semantic transfer from consecutive images.

Keywords

Consecutive images, High-fidelity, Image editing, Inversion methods, Inversion process, Joint inversion, Property, Real images, State-of-the-art methods, Video frame

Discipline

Databases and Information Systems

Research Areas

Information Systems and Management

Publication

Proceedings of the 18th IEEE/CVF International Conference on Computer Vision, Virtual, Online, 2021 October 11-17

First Page

13890

Last Page

13898

ISBN

9781665428125

Identifier

10.1109/ICCV48922.2021.01365

Publisher

IEEE

City or Country

New Jersey

Additional URL

https://doi.org/10.1109/ICCV48922.2021.01365

Download

Share

COinS