Publication Type
Journal Article
Version
publishedVersion
Publication Date
2-2020
Abstract
We propose a novel deep example-based image colourization method called dense encoding pyramid network. In our study, we define the colourization as a multinomial classification problem. Given a greyscale image and a reference image, the proposed network leverages large-scale data and then predicts colours by analysing the colour distribution of the reference image. We design the network as a pyramid structure in order to exploit the inherent multi-scale, pyramidal hierarchy of colour representations. Between two adjacent levels, we propose a hierarchical decoder–encoder filter to pass the colour distributions from the lower level to higher level in order to take both semantic information and fine details into account during the colourization process. Within the network, a novel parallel residual dense block is proposed to effectively extract the local–global context of the colour representations by widening the network. Several experiments, as well as a user study, are conducted to evaluate the performance of our network against state-of-the-art colourization methods. Experimental results show that our network is able to generate colourful, semantically correct and visually pleasant colour images. In addition, unlike fully automatic colourization that produces fixed colour images, the reference image of our network is flexible; both natural images and simple colour palettes can be used to guide the colourization. © 2019 The Authors Computer Graphics Forum
Keywords
Computational photography, Computing methodologies, Computing methodology, neural network, Example based, I.3.3 [computer graphics], I.3.3 [computer graphic], picture/image, Image and video processing, Images processing, Neural-networks, Reference image
Discipline
Graphics and Human Computer Interfaces
Research Areas
Information Systems and Management
Publication
Computer Graphics Forum
Volume
39
Issue
1
First Page
20
Last Page
33
ISSN
0167-7055
Identifier
10.1111/cgf.13659
Publisher
Wiley
Citation
XIAO, Chufeng; HAN, Chu; ZHANG, Zhuming; QIN, Jing; WONG, Tien-Tsin; HAN, Guoqiang; and HE, Shengfeng.
Example-based colourization via dense encoding pyramids. (2020). Computer Graphics Forum. 39, (1), 20-33.
Available at: https://ink.library.smu.edu.sg/sis_research/8361
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1111/cgf.13659