Publication Type
Journal Article
Version
acceptedVersion
Publication Date
7-2024
Abstract
Restoring old photographs can preserve cherished memories. Previous methods handled diverse damages within the same network structure, which proved impractical. In addition, these methods cannot exploit correlations among artifacts, especially in scratches versus patch-misses issues. Hence, a tailored network is particularly crucial. In light of this, we propose a unified framework consisting of two key components: ScratchNet and PatchNet. In detail, ScratchNet employs the parallel Multi-scale Partial Convolution Module to effectively repair scratches, learning from multi-scale local receptive fields. In contrast, the patch-misses necessitate the network to emphasize global information. To this end, we incorporate a transformer-based encoder and decoder architecture. In the encoder phase, we introduce a Non-local Inpainting Attention Module, replacing the multi-head attention, to facilitate holistic context inpainting. In the decoder phase, the Mask-aware Instance Norm Module replaces the Layer Normalization, ensuring style consistency between foreground and background. Finally, the outcomes of ScratchNet are integrated into the PatchNet pipeline to supplement contextual information hierarchically. Mining damage correlations assists in training the network in an easy-to-hard manner. Extensive experiments demonstrate the superiority of our method over state-of-the-art approaches. The code is available at https://github.com/cwyyt/Hierarchical-Damage-Correlations-for-OldPhoto-Restoration.
Keywords
Image inpainting, Old photo restoration, Transformer
Discipline
Graphics and Human Computer Interfaces | Software Engineering
Research Areas
Software and Cyber-Physical Systems
Publication
Information Fusion
Volume
107
First Page
1
Last Page
11
ISSN
1566-2535
Identifier
10.1016/j.inffus.2024.102340
Publisher
Elsevier
Citation
CAI, Weiwei; XU, Xuemiao; XU, Jiajia; ZHANG, Huaidong; YANG, Haoxin; ZHANG, Kun; and HE, Shengfeng.
Hierarchical damage correlations for old photo restoration. (2024). Information Fusion. 107, 1-11.
Available at: https://ink.library.smu.edu.sg/sis_research/8730
Copyright Owner and License
Authors
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1016/j.inffus.2024.102340