Multi-level Personalized Federated Learning on Heterogeneous and Long-Tailed Data
<|reference_start|>Multi-level Personalized Federated Learning on Heterogeneous and Long-Tailed Data: Federated learning (FL) offers a privacy-centric distributed learning framework, enabling model training on individual clients and central aggregation without necessitating data exchange. Nonetheless, FL implementations often suffer from non-i.i.d. and long-tailed class distributions across mobile applications, e.g., autonomous vehicles, which leads models to overfitting as local training may converge to sub-optimal. In our study, we explore the impact of data heterogeneity on model bias and introduce an innovative personalized FL framework, Multi-level Personalized Federated Learning (MuPFL), which leverages the hierarchical architecture of FL to fully harness computational resources at various levels. This framework integrates three pivotal modules: Biased Activation Value Dropout (BAVD) to mitigate overfitting and accelerate training; Adaptive Cluster-based Model Update (ACMU) to refine local models ensuring coherent global aggregation; and Prior Knowledge-assisted Classifier Fine-tuning (PKCF) to bolster classification and personalize models in accord with skewed local data with shared knowledge. Extensive experiments on diverse real-world datasets for image classification and semantic segmentation validate that MuPFL consistently outperforms state-of-the-art baselines, even under extreme non-i.i.d. and long-tail conditions, which enhances accuracy by as much as 7.39% and accelerates training by up to 80% at most, marking significant advancements in both efficiency and effectiveness.<|reference_end|>
@article{zhang2024multi-level,
    title={Multi-level Personalized Federated Learning on Heterogeneous and
  Long-Tailed Data},
    author={Rongyu Zhang, Yun Chen, Chenrui Wu, Fangxin Wang, Bo Li},
    journal={arXiv preprint arXiv:2405.06413},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.06413},
    primaryClass={cs.AI}
}