Contextual-assisted scratched photo restoration
Publication Type
Journal Article
Publication Date
3-2023
Abstract
Printed photographs can be easily warped, wrinkled, and even deteriorated over time. Existing methods treat the restoration of scratches as a pure inpainting problem that neglects the underlying corrupted contextual knowledge. However, important underlying contents are hidden behind the scratches, which are essential hints for producing a semantically consistent result. Motivated by this insight, we explore how to harmonize the scratch-free features and noisy but essential scratch features to produce a visually consistent restoration. Specifically, in this paper, we propose an automatic retouching approach for scratched photographs with the aid of scratch/background context. We explicitly process scratch and background context in two stages. In the first stage, we mainly extract global scratch features, while the mask is introduced in the second stage to filter out and inpaint the scratches. Both contexts are carefully reciprocated for a faithful restoration. Particularly, we propose a Scratch Contextual Assisted Module (SCAM) to adaptively learn texture within the detected mask. This module utilizes the distance between the scratch mask-out feature and scratch encoder feature for modeling the pixel-wise correspondence, which determines the importance of the encoder feature within the scratch mask. Furthermore, to facilitate the evaluation of scratch restoration methods, we create two new scratched photo datasets which have 238 scratch/scratch-free photo pairs to promote the development in the scratch restoration field, namely Old Scratched Photo Dataset (OSPD) and Modern Scratched Photo Dataset (MSPD). Extensive experimental results on the proposed datasets demonstrate that our model outperforms existing methods. To extend the application, we also perform the proposed method on video samples and obtain visual-pleasing results. The code can be found at https://github.com/cwyyt/Contextual-assisted-Scratched-Photo-Restoration
Keywords
Image restoration, Feature extraction, Deep learning, Electronic mail, Context modeling, Videos, Task analysis
Discipline
Computational Engineering | Graphics and Human Computer Interfaces
Research Areas
Data Science and Engineering
Publication
IEEE Transactions on Circuits and Systems for Video Technology
Volume
33
Issue
10
First Page
5458
Last Page
5469
ISSN
1051-8215
Identifier
10.1109/TCSVT.2023.3256372
Publisher
Institute of Electrical and Electronics Engineers
Citation
CAI, Weiwei; ZHANG, Huaidong; XU, Xuemiao; HE, Shengfeng; ZHANG, Kun; and QIN, Jing.
Contextual-assisted scratched photo restoration. (2023). IEEE Transactions on Circuits and Systems for Video Technology. 33, (10), 5458-5469.
Available at: https://ink.library.smu.edu.sg/sis_research/8377
Additional URL
https://doi.org/10.1109/TCSVT.2023.3256372