Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
8-2022
Abstract
Data collected by IoT devices are often private and have a large diversity across users. Therefore, learning requires pre-training a model with available representative data samples, deploying the pre-trained model on IoT devices, and adapting the deployed model on the device with local data. Such an on-device adaption for deep learning empowered applications demands data and memory efficiency. However, existing gradient-based meta learning schemes fail to support memory-efficient adaptation. To this end, we propose p-Meta, a new meta learning method that enforces structure-wise partial parameter updates while ensuring fast generalization to unseen tasks. Evaluations on few-shot image classification and reinforcement learning tasks show that p-Meta not only improves the accuracy but also substantially reduces the peak dynamic memory by a factor of 2.5 on average compared to state-of-the-art few-shot adaptation methods.
Keywords
deep neural networks, memory-efficient training, meta learning
Discipline
Numerical Analysis and Scientific Computing
Research Areas
Software and Cyber-Physical Systems
Publication
KDD '22: n Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, August 14-18
First Page
1441
Last Page
1451
ISBN
9781450393850
Identifier
10.1145/3534678.3539293
Publisher
ACM
City or Country
New York
Citation
QU, Zhongnan; ZHOU, Zimu; TONG, Yongxin; and THIELE, Lothar.
p-Meta: Towards on-device deep model adaptation. (2022). KDD '22: n Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, August 14-18. 1441-1451.
Available at: https://ink.library.smu.edu.sg/sis_research/7275
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1145/3534678.3539293
