Publication Type

Conference Proceeding Article

Version

acceptedVersion

Publication Date

8-2024

Abstract

Although Federated Learning (FL) enables global model training Xiaofei Xie xfxie@smu.edu.sg Singapore Management University Singapore, Singapore Xian Wei xwei@sei.ecnu.edu.cn East China Normal University Shanghai, China Mingsong Chen∗ mschen@sei.ecnu.edu.cn East China Normal University Shanghai, China • Computing methodologies → Distributed artificial intelligence. across clients without compromising their raw data, due to the unevenly distributed data among clients, existing Federated Averaging (FedAvg)-based methods suffer from the problem of low inference performance. Specifically, different data distributions among clients lead to various optimization directions of local models. Aggregating local models usually results in a low-generalized global model, which performs worse on most of the clients. To address the above issue, inspired by the observation from a geometric perspective that a well-generalized solution is located in a flat area rather than a sharp area, we propose a novel and heuristic FL paradigm named FedMR (Federated Model Recombination). The goal of FedMR is to guide the recombined models to be trained towards a flat area. Unlike conventional FedAvg-based methods, in FedMR, the cloud server recombines collected local models by shuffling each layer of them to generate multiple recombined models for local training on clients rather than an aggregated global model. Since the area of the f lat area is larger than the sharp area, when local models are located in different areas, recombined models have a higher probability of locating in a flat area. When all recombined models are located in the same flat area, they are optimized towards the same direction. Wetheoretically analyze the convergence of model recombination. Experimental results show that, compared with state-of-the-art FL methods, FedMR can significantly improve the inference accuracy without exposing the privacy of each client.

Keywords

Federated learning, Model recombination, Non-IID, Generalization

Discipline

Artificial Intelligence and Robotics | Databases and Information Systems

Research Areas

Data Science and Engineering; Information Systems and Management

Publication

Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Barcelona, Spain, 2024 August 25–29

ISBN

9798400704901

Identifier

10.1145/3637528.3671722

Publisher

ACM

City or Country

New York

Additional URL

https://doi.org/10.1145/3637528.3671722

Download

Share

COinS