FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models
<|reference_start|>FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models: Federated Learning (FL) has garnered increasing attention due to its unique characteristic of allowing heterogeneous clients to process their private data locally and interact with a central server, while being respectful of privacy. A critical bottleneck in FL is the communication cost. A pivotal strategy to mitigate this burden is \emph{Local Training}, which involves running multiple local stochastic gradient descent iterations between communication phases. Our work is inspired by the innovative \emph{Scaffnew} algorithm, which has considerably advanced the reduction of communication complexity in FL. We introduce FedComLoc (Federated Compressed and Local Training), integrating practical and effective compression into \emph{Scaffnew} to further enhance communication efficiency. Extensive experiments, using the popular TopK compressor and quantization, demonstrate its prowess in substantially reducing communication overheads in heterogeneous settings.<|reference_end|>
@article{yi2024fedcomloc:,
    title={FedComLoc: Communication-Efficient Distributed Training of Sparse and
  Quantized Models},
    author={Kai Yi, Georg Meinhardt, Laurent Condat, Peter Richt'arik},
    journal={arXiv preprint arXiv:2403.09904},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.09904},
    primaryClass={cs.LG cs.AI cs.DC}
}