Publication Type

Journal Article

Version

acceptedVersion

Publication Date

8-2023

Abstract

Synthesizing novel views from a single view image is a highly ill-posed problem. We discover an effective solution to reduce the learning ambiguity by expanding the single-view view synthesis problem to a multi-view setting. Specifically, we leverage the reliable and explicit stereo prior to generate a pseudo-stereo viewpoint, which serves as an auxiliary input to construct the 3D space. In this way, the challenging novel view synthesis process is decoupled into two simpler problems of stereo synthesis and 3D reconstruction. In order to synthesize a structurally correct and detail-preserved stereo image, we propose a self-rectified stereo synthesis to amend erroneous regions in an identify-rectify manner. Hard-to-train and incorrect warping samples are first discovered by two strategies, (1) pruning the network to reveal low-confident predictions; and (2) bidirectionally matching between stereo images to allow the discovery of improper mapping. These regions are then inpainted to form the final pseudo-stereo. With the aid of this extra input, a preferable 3D reconstruction can be easily obtained, and our method can work with arbitrary 3D representations. Extensive experiments show that our method outperforms state-of-the-art single-view view synthesis methods and stereo synthesis methods.

Keywords

3D reconstruction, Effective solution, Ill posed problem, Multi-views, Pseudo stereos, Stereo synthesis, Stereoimages, Synthesis method, Synthesis problems, View synthesis

Discipline

Databases and Information Systems

Research Areas

Information Systems and Management

Publication

International Journal of Computer Vision

Volume

131

Issue

8

First Page

2032

Last Page

2043

ISSN

0920-5691

Identifier

10.1007/s11263-023-01803-z

Publisher

Springer

Copyright Owner and License

Authors

Additional URL

https://doi.org/10.1007/s11263-023-01803-z

Download

Share

COinS