FedBAT: Communication-Efficient Federated Learning via Learnable Binarization
<|reference_start|>FedBAT: Communication-Efficient Federated Learning via Learnable Binarization: Federated learning is a promising distributed machine learning paradigm that can effectively exploit large-scale data without exposing users' privacy. However, it may incur significant communication overhead, thereby potentially impairing the training efficiency. To address this challenge, numerous studies suggest binarizing the model updates. Nonetheless, traditional methods usually binarize model updates in a post-training manner, resulting in significant approximation errors and consequent degradation in model accuracy. To this end, we propose Federated Binarization-Aware Training (FedBAT), a novel framework that directly learns binary model updates during the local training process, thus inherently reducing the approximation errors. FedBAT incorporates an innovative binarization operator, along with meticulously designed derivatives to facilitate efficient learning. In addition, we establish theoretical guarantees regarding the convergence of FedBAT. Extensive experiments are conducted on four popular datasets. The results show that FedBAT significantly accelerates the convergence and exceeds the accuracy of baselines by up to 9\%, even surpassing that of FedAvg in some cases.<|reference_end|>
@article{li2024fedbat:,
    title={FedBAT: Communication-Efficient Federated Learning via Learnable
  Binarization},
    author={Shiwei Li and Wenchao Xu and Haozhao Wang and Xing Tang and Yining Qi
  and Shijie Xu and Weihong Luo and Yuhua Li and Xiuqiang He and Ruixuan Li},
    journal={arXiv preprint arXiv:2408.03215},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.03215},
    primaryClass={cs.LG cs.DC}
}