Personalized Federated Learning on Heterogeneous and Long-Tailed Data via Expert Collaborative Learning
<|reference_start|>Personalized Federated Learning on Heterogeneous and Long-Tailed Data via Expert Collaborative Learning: Personalized Federated Learning (PFL) aims to acquire customized models for each client without disclosing raw data by leveraging the collective knowledge of distributed clients. However, the data collected in real-world scenarios is likely to follow a long-tailed distribution. For example, in the medical domain, it is more common for the number of general health notes to be much larger than those specifically relatedto certain diseases. The presence of long-tailed data can significantly degrade the performance of PFL models. Additionally, due to the diverse environments in which each client operates, data heterogeneity is also a classic challenge in federated learning. In this paper, we explore the joint problem of global long-tailed distribution and data heterogeneity in PFL and propose a method called Expert Collaborative Learning (ECL) to tackle this problem. Specifically, each client has multiple experts, and each expert has a different training subset, which ensures that each class, especially the minority classes, receives sufficient training. Multiple experts collaborate synergistically to produce the final prediction output. Without special bells and whistles, the vanilla ECL outperforms other state-of-the-art PFL methods on several benchmark datasets under different degrees of data heterogeneity and long-tailed distribution.<|reference_end|>
@article{lv2024personalized,
    title={Personalized Federated Learning on Heterogeneous and Long-Tailed Data
  via Expert Collaborative Learning},
    author={Fengling Lv and Xinyi Shang and Yang Zhou and Yiqun Zhang and Mengke
  Li and Yang Lu},
    journal={arXiv preprint arXiv:2408.02019},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.02019},
    primaryClass={cs.LG}
}