Publication Type
Conference Proceeding Article
Version
acceptedVersion
Publication Date
10-2021
Abstract
Data augmentation has been an indispensable tool to improve the performance of deep neural networks, however the augmentation can hardly transfer among different tasks and datasets. Consequently, a recent trend is to adopt AutoML technique to learn proper augmentation policy without extensive hand-crafted tuning. In this paper, we propose an efficient differentiable search algorithm called Direct Differentiable Augmentation Search (DDAS). It exploits meta-learning with one-step gradient update and continuous relaxation to the expected training loss for efficient search. Our DDAS can achieve efficient augmentation search without relying on approximations such as Gumbel-Softmax or second order gradient approximation. To further reduce the adverse effect of improper augmentations, we organize the search space into a two level hierarchy, in which we first decide whether to apply augmentation, and then determine the specific augmentation policy. On standard image classification benchmarks, our DDAS achieves state-of-the-art performance and efficiency tradeoff while reducing the search cost dramatically, e.g. 0.15 GPU hours for CIFAR-10. In addition, we also use DDAS to search augmentation for object detection task and achieve comparable performance with AutoAugment [8], while being 1000× faster
Discipline
Databases and Information Systems
Research Areas
Data Science and Engineering
Publication
2021 ICCV Virtual Oct 11-17
First Page
12219
Last Page
12228
Publisher
IEEE Computer Society
City or Country
Virtual
Citation
LIU, Aoming; HUANG, Zehao; HUANG, Zhiwu; Huang; and WANG, Naiyan.
Direct differentiable augmentation search. (2021). 2021 ICCV Virtual Oct 11-17. 12219-12228.
Available at: https://ink.library.smu.edu.sg/sis_research/6261
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.