Publication Type
Conference Proceeding Article
Version
acceptedVersion
Publication Date
10-2023
Abstract
Video inpainting aims at filling in missing regions of a video. However, when dealing with dynamic scenes with camera or object movements, annotating the inpainting target becomes laborious and impractical. In this paper, we resolve the one-shot video inpainting problem in which only one annotated first frame is provided. A naive solution is to propagate the initial target to the other frames with techniques like object tracking. In this context, the main obstacles are the unreliable propagation and the partially inpainted artifacts due to the inaccurate mask. For the former problem, we propose curricular inactivation to replace the hard masking mechanism for indicating the inpainting target, which is robust to erroneous predictions in long-term video inpainting. For the latter, we explore the properties of inpainting residue and present an online residue removal method in an iterative detect-and-refine manner. Extensive experiments on several real-world datasets demonstrate the quantitative and qualitative superiorities of our proposed method in one-shot video inpainting. More importantly, our method is extremely flexible that can be integrated with arbitrary traditional inpainting models, activating them to perform the reliable one-shot video inpainting task. Video demonstrations can be found in our supplement, and our code can be found at https://github.com/Arise-zwy/CIRI.
Keywords
Video inpainting, Curricular Inactivation, CIRI
Discipline
Computer Sciences | Graphics and Human Computer Interfaces
Research Areas
Software and Cyber-Physical Systems
Publication
2023 IEEE/CVF International Conference on Computer Vision (ICCV): Paris, October 2-6: Proceedings
First Page
13012
Last Page
13022
ISBN
9798350307184
Identifier
10.1109/ICCV51070.2023.01196
Publisher
IEE
City or Country
Piscataway, NJ
Citation
ZHENG, Weiying; XU, Cheng; XU, Xuemiao; LIU, Wenxi; and HE, Shengfeng.
CIRI: Curricular inactivation for residue-aware one-shot video inpainting. (2023). 2023 IEEE/CVF International Conference on Computer Vision (ICCV): Paris, October 2-6: Proceedings. 13012-13022.
Available at: https://ink.library.smu.edu.sg/sis_research/8535
Copyright Owner and License
Authors
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1109/ICCV51070.2023.01196