Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
2-2021
Abstract
This paper aims at enlarging the problem of Neural Architecture Search (NAS) from Single-Path and Multi-Path Search to automated Mixed-Path Search. In particular, we model the NAS problem as a sparse supernet using a new continuous architecture representation with a mixture of sparsity constraints. The sparse supernet enables us to automatically achieve sparsely-mixed paths upon a compact set of nodes. To optimize the proposed sparse supernet, we exploit a hierarchical accelerated proximal gradient algorithm within a bi-level optimization framework. Extensive experiments on Convolutional Neural Network and Recurrent Neural Network search demonstrate that the proposed method is capable of searching for compact, general and powerful neural architectures.
Discipline
OS and Networks | Systems Architecture
Research Areas
Data Science and Engineering
Publication
The Proceedings of 35th AAAI Conference on Artificial Intelligence, 2021 Feb 2-9
First Page
10379
Last Page
10387
Publisher
AAAI Press
City or Country
Virtual
Citation
WU, Y.; LIU, A.; HUANG, Zhiwu; ZHANG, S.; and VAN, Gool L..
Neural architecture search as sparse supernet. (2021). The Proceedings of 35th AAAI Conference on Artificial Intelligence, 2021 Feb 2-9. 10379-10387.
Available at: https://ink.library.smu.edu.sg/sis_research/6411
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.