KoReA-SFL: Knowledge Replay-based Split Federated Learning Against Catastrophic Forgetting
<|reference_start|>KoReA-SFL: Knowledge Replay-based Split Federated Learning Against Catastrophic Forgetting: Although Split Federated Learning (SFL) is good at enabling knowledge sharing among resource-constrained clients, it suffers from the problem of low training accuracy due to the neglect of data heterogeneity and catastrophic forgetting. To address this issue, we propose a novel SFL approach named KoReA-SFL, which adopts a multi-model aggregation mechanism to alleviate gradient divergence caused by heterogeneous data and a knowledge replay strategy to deal with catastrophic forgetting. Specifically, in KoReA-SFL cloud servers (i.e., fed server and main server) maintain multiple branch model portions rather than a global portion for local training and an aggregated master-model portion for knowledge sharing among branch portions. To avoid catastrophic forgetting, the main server of KoReA-SFL selects multiple assistant devices for knowledge replay according to the training data distribution of each server-side branch-model portion. Experimental results obtained from non-IID and IID scenarios demonstrate that KoReA-SFL significantly outperforms conventional SFL methods (by up to 23.25\% test accuracy improvement).<|reference_end|>
@article{xia2024korea-sfl:,
    title={KoReA-SFL: Knowledge Replay-based Split Federated Learning Against
  Catastrophic Forgetting},
    author={Zeke Xia and Ming Hu and Dengke Yan and Ruixuan Liu and Anran Li and
  Xiaofei Xie and Mingsong Chen},
    journal={arXiv preprint arXiv:2404.12846},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.12846},
    primaryClass={cs.LG}
}