Delving into important samples of semi-supervised old photo restoration: A new dataset and method
Publication Type
Journal Article
Publication Date
1-2024
Abstract
The degradation of printed photographs due to inadequate preservation is a major problem that can be addressed through deep learning-based restoration methods. However, these methods are often limited by their reliance on annotated data, making them less effective for new domains with limited training samples. In this paper, we propose a semi-supervised old photo restoration network that employs a continuous important sample mining strategy. Specifically, we explore the learning potential of limited data from three aspects: correcting imbalanced data distribution, assigning significant pseudo labels, and learning from unlabeled data. First, we coordinate a random mask augmented strategy with the Double-consistency Alignment method to address the unbalanced damaged category (scratched damage is more prevalent than other artifact types). Second, we develop a novel Perceptual-aware Pseudo-label Propagation method that selects initial recovered results as reliable pseudo-labels to continuously expand the sample pool. Lastly, we propose a Damage-augmented Contrastive Learning method that constructs positive, anchor, and negative samples within a semi-supervised framework to mine correlations of unlabeled data more effectively. To evaluate our approach, we introduce the Old Photo Detection Dataset () and the Old Photo Restoration Dataset (), both of which consist of 563 (6,179 augmented) photo pairs recovered by professional artists. Our extensive experiments show that our approach significantly outperforms existing methods. Furthermore, we demonstrate the effectiveness of our approach by training an external old photographic plate restoration network using the deuterogenic old photographic film dataset and obtaining promising results.
Keywords
contrastive learning, Correlation, Image restoration, Old photo restoration, Reliability, semi-supervised learning, Semisupervised learning, Task analysis, Training data
Discipline
Graphics and Human Computer Interfaces | Software Engineering
Research Areas
Software and Cyber-Physical Systems
Publication
IEEE Transactions on Multimedia
First Page
1
Last Page
13
ISSN
1520-9210
Identifier
10.1109/TMM.2024.3400695
Publisher
Institute of Electrical and Electronics Engineers
Citation
CAI, Werwei; ZHANG, Huaidong; XU, Xuemiao; XU, Chenshu; ZHANG, Kun; and HE, Shengfeng.
Delving into important samples of semi-supervised old photo restoration: A new dataset and method. (2024). IEEE Transactions on Multimedia. 1-13.
Available at: https://ink.library.smu.edu.sg/sis_research/8820
Additional URL
https://doi.org/10.1109/TMM.2024.3400695