Publication Type
Journal Article
Version
acceptedVersion
Publication Date
12-2024
Abstract
Along with the increasing popularity of Deep Learning (DL) techniques, more and more Artificial Intelligence of Things (AIoT) systems are adopting federated learning (FL) to enable privacy-aware collaborative learning among AIoT devices. However, due to the inherent data and device heterogeneity issues, existing FL-based AIoT systems suffer from the model selection problem. Although various heterogeneous FL methods have been investigated to enable collaborative training among heterogeneous models, there is still a lack of i) wise heterogeneous model generation methods for devices, ii) consideration of uncertain factors, and iii) performance guarantee for large models, thus strongly limiting the overall FL performance. To address the above issues, this paper introduces a novel heterogeneous FL framework named FlexFL. By adopting our Average Percentage of Zeros (APoZ)-guided flexible pruning strategy, FlexFL can effectively derive best-fit models for heterogeneous devices to explore their greatest potential. Meanwhile, our proposed adaptive local pruning strategy allows AIoT devices to prune their received models according to their varying resources within uncertain scenarios. Moreover, based on self-knowledge distillation, FlexFL can enhance the inference performance of large models by learning knowledge from small models. Comprehensive experimental results show that, compared to state-of-the-art heterogeneous FL methods, FlexFL can significantly improve the overall inference accuracy by up to 14.24%.
Keywords
AIoT, APoZ, Heterogeneous federated learning, Model pruning, Uncertain scenario
Discipline
Artificial Intelligence and Robotics | Computer Sciences
Research Areas
Data Science and Engineering; Intelligent Systems and Optimization
Areas of Excellence
Digital transformation
Publication
IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Volume
43
Issue
11
First Page
4069
Last Page
4080
ISSN
02780070
Identifier
10.1109/TCAD.2024.3444695
Publisher
Institute of Electrical and Electronics Engineers
Citation
CHEN, Zekai; JIA, Chentao; HU, Ming; XIE, Xiaofei; LI, Anran; and CHEN, Mingsong.
FlexFL: Heterogeneous federated learning via APoZ-guided flexible pruning in uncertain scenarios. (2024). IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems. 43, (11), 4069-4080.
Available at: https://ink.library.smu.edu.sg/sis_research/9817
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1109/TCAD.2024.3444695