Deep Unsupervised Pixelization

Author

Chu HAN
Qiang WEN
Shengfeng HE, Singapore Management UniversityFollow
Qianshu ZHU
Yinjie TAN
Guoqiang HAN
Tien-Tsin WONG

Publication Type

Journal Article

Version

publishedVersion

Publication Date

12-2018

Abstract

In this paper, we present a novel unsupervised learning method for pixelization. Due to the difficulty in creating pixel art, preparing the paired training data for supervised learning is impractical. Instead, we propose an unsupervised learning framework to circumvent such difficulty. We leverage the dual nature of the pixelization and depixelization, and model these two tasks in the same network in a bi-directional manner with the input itself as training supervision. These two tasks are modeled as a cascaded network which consists of three stages for different purposes. GridNet transfers the input image into multi-scale grid-structured images with different aliasing effects. PixelNet associated with GridNet to synthesize pixel arts with sharp edges and perceptually optimal local structures. DepixelNet connects the previous network and aims to recover the pixelized result to the original image. For the sake of unsupervised learning, the mirror loss is proposed to hold the reversibility of feature representations in the process. In addition, adversarial, L1, and gradient losses are involved in the network to obtain pixel arts by retaining color correctness and smoothness. We show that our technique can synthesize crisper and perceptually more appropriate pixel arts than state-of-the-art image downscaling methods. We evaluate the proposed method with extensive experiments on many images. The proposed method outperforms state-of-the-art methods in terms of visual quality and user preference.

Keywords

Adversarial networks, Downscaling methods, Feature representation, Image translation, Pixelization, State-of-the-art methods, Structured images, Unsupervised learning method

Discipline

Databases and Information Systems | Digital Communications and Networking

Research Areas

Information Systems and Management

Publication

ACM Transactions on Graphics

Volume

37

Issue

6

ISSN

0730-0301

Identifier

10.1145/3272127.3275082

Publisher

Association for Computing Machinery (ACM)

Citation

HAN, Chu; WEN, Qiang; HE, Shengfeng; ZHU, Qianshu; TAN, Yinjie; HAN, Guoqiang; and WONG, Tien-Tsin. Deep Unsupervised Pixelization. (2018). ACM Transactions on Graphics. 37, (6),.
Available at: https://ink.library.smu.edu.sg/sis_research/8449

Copyright Owner and License

Authors

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.

Additional URL

https://doi.org/10.1145/3272127.3275082