CaBaFL: Asynchronous federated learning via hierarchical cache and feature balance
Publication Type
Journal Article
Publication Date
11-2024
Abstract
Federated learning (FL) as a promising distributed machine learning paradigm has been widely adopted in Artificial Intelligence of Things (AIoT) applications. However, the efficiency and inference capability of FL is seriously limited due to the presence of stragglers and data imbalance across massive AIoT devices, respectively. To address the above challenges, we present a novel asynchronous FL approach named CaBaFL, which includes a hierarchical cache-based aggregation mechanism and a feature balance-guided device selection strategy. CaBaFL maintains multiple intermediate models simultaneously for local training. The hierarchical cache-based aggregation mechanism enables each intermediate model to be trained on multiple devices to align the training time and mitigate the straggler issue. In specific, each intermediate model is stored in a low-level cache for local training and when it is trained by sufficient local devices, it will be stored in a high-level cache for aggregation. To address the problem of imbalanced data, the feature balance-guided device selection strategy in CaBaFL adopts the activation distribution as a metric, which enables each intermediate model to be trained across devices with totally balanced data distributions before aggregation. Experimental results show that compared to the state-of-the-art FL methods, CaBaFL achieves up to 9.26X training acceleration and 19.71% accuracy improvements.
Keywords
Artificial Intelligence of Things AIoT), asynchronous federated learning, data/device heterogeneity, feature balance
Discipline
Software Engineering
Research Areas
Software and Cyber-Physical Systems
Publication
IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Volume
43
Issue
11
First Page
4057
Last Page
4068
ISSN
0278-0070
Identifier
10.1109/TCAD.2024.3446881
Publisher
Institute of Electrical and Electronics Engineers
Citation
XIA, Zeke; HU, Ming; YAN, Dengke; XIE, Xiaofei; LI, Tianlin; LI, Anran; ZHOU, Junlong; and CHEN, Mingsong.
CaBaFL: Asynchronous federated learning via hierarchical cache and feature balance. (2024). IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems. 43, (11), 4057-4068.
Available at: https://ink.library.smu.edu.sg/sis_research/9563
Additional URL
https://doi.org/10.1109/TCAD.2024.3446881