Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

8-2022

Abstract

Data collected by IoT devices are often private and have a large diversity across users. Therefore, learning requires pre-training a model with available representative data samples, deploying the pre-trained model on IoT devices, and adapting the deployed model on the device with local data. Such an on-device adaption for deep learning empowered applications demands data and memory efficiency. However, existing gradient-based meta learning schemes fail to support memory-efficient adaptation. To this end, we propose p-Meta, a new meta learning method that enforces structure-wise partial parameter updates while ensuring fast generalization to unseen tasks. Evaluations on few-shot image classification and reinforcement learning tasks show that p-Meta not only improves the accuracy but also substantially reduces the peak dynamic memory by a factor of 2.5 on average compared to state-of-the-art few-shot adaptation methods.

Keywords

deep neural networks, memory-efficient training, meta learning

Discipline

Numerical Analysis and Scientific Computing

Research Areas

Software and Cyber-Physical Systems

Publication

KDD '22: n Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, August 14-18

First Page

1441

Last Page

1451

ISBN

9781450393850

Identifier

10.1145/3534678.3539293

Publisher

ACM

City or Country

New York

Additional URL

https://doi.org/10.1145/3534678.3539293

Download

Share

COinS