Contextual-assisted scratched photo restoration

Publication Type

Journal Article

Publication Date

3-2023

Abstract

Printed photographs can be easily warped, wrinkled, and even deteriorated over time. Existing methods treat the restoration of scratches as a pure inpainting problem that neglects the underlying corrupted contextual knowledge. However, important underlying contents are hidden behind the scratches, which are essential hints for producing a semantically consistent result. Motivated by this insight, we explore how to harmonize the scratch-free features and noisy but essential scratch features to produce a visually consistent restoration. Specifically, in this paper, we propose an automatic retouching approach for scratched photographs with the aid of scratch/background context. We explicitly process scratch and background context in two stages. In the first stage, we mainly extract global scratch features, while the mask is introduced in the second stage to filter out and inpaint the scratches. Both contexts are carefully reciprocated for a faithful restoration. Particularly, we propose a Scratch Contextual Assisted Module (SCAM) to adaptively learn texture within the detected mask. This module utilizes the distance between the scratch mask-out feature and scratch encoder feature for modeling the pixel-wise correspondence, which determines the importance of the encoder feature within the scratch mask. Furthermore, to facilitate the evaluation of scratch restoration methods, we create two new scratched photo datasets which have 238 scratch/scratch-free photo pairs to promote the development in the scratch restoration field, namely Old Scratched Photo Dataset (OSPD) and Modern Scratched Photo Dataset (MSPD). Extensive experimental results on the proposed datasets demonstrate that our model outperforms existing methods. To extend the application, we also perform the proposed method on video samples and obtain visual-pleasing results. The code can be found at https://github.com/cwyyt/Contextual-assisted-Scratched-Photo-Restoration

Keywords

Image restoration, Feature extraction, Deep learning, Electronic mail, Context modeling, Videos, Task analysis

Discipline

Computational Engineering | Graphics and Human Computer Interfaces

Research Areas

Data Science and Engineering

Publication

IEEE Transactions on Circuits and Systems for Video Technology

Volume

33

Issue

10

First Page

5458

Last Page

5469

ISSN

1051-8215

Identifier

10.1109/TCSVT.2023.3256372

Publisher

Institute of Electrical and Electronics Engineers

Additional URL

https://doi.org/10.1109/TCSVT.2023.3256372

This document is currently not available here.

Share

COinS