Unmasking Efficiency: Learning Salient Sparse Models in Non-IID Federated Learning
<|reference_start|>Unmasking Efficiency: Learning Salient Sparse Models in Non-IID Federated Learning: In this work, we propose Salient Sparse Federated Learning (SSFL), a streamlined approach for sparse federated learning with efficient communication. SSFL identifies a sparse subnetwork prior to training, leveraging parameter saliency scores computed separately on local client data in non-IID scenarios, and then aggregated, to determine a global mask. Only the sparse model weights are communicated each round between the clients and the server. We validate SSFL's effectiveness using standard non-IID benchmarks, noting marked improvements in the sparsity--accuracy trade-offs. Finally, we deploy our method in a real-world federated learning framework and report improvement in communication time.<|reference_end|>
@article{ohib2024unmasking,
    title={Unmasking Efficiency: Learning Salient Sparse Models in Non-IID
  Federated Learning},
    author={Riyasat Ohib, Bishal Thapaliya, Gintare Karolina Dziugaite, Jingyu
  Liu, Vince Calhoun and Sergey Plis},
    journal={arXiv preprint arXiv:2405.09037},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.09037},
    primaryClass={cs.LG cs.AI cs.DC}
}