OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learning
<|reference_start|>OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learning: Trained on massive publicly available data, large language models (LLMs) have demonstrated tremendous success across various fields. While more data contributes to better performance, a disconcerting reality is that high-quality public data will be exhausted in a few years. In this paper, we offer a potential next step for contemporary LLMs: collaborative and privacy-preserving LLM training on the underutilized distributed private data via federated learning (FL), where multiple data owners collaboratively train a shared model without transmitting raw data. To achieve this, we build a concise, integrated, and research-friendly framework/codebase, named OpenFedLLM. It covers federated instruction tuning for enhancing instruction-following capability, federated value alignment for aligning with human values, and 7 representative FL algorithms. Besides, OpenFedLLM supports training on diverse domains, where we cover 8 training datasets; and provides comprehensive evaluations, where we cover 30+ evaluation metrics. Through extensive experiments, we observe that all FL algorithms outperform local training on training LLMs, demonstrating a clear performance improvement across a variety of settings. Notably, in a financial benchmark, Llama2-7B fine-tuned by applying any FL algorithm can outperform GPT-4 by a significant margin while the model obtained through individual training cannot, demonstrating strong motivation for clients to participate in FL. The code is available at https://github.com/rui-ye/OpenFedLLM.<|reference_end|>
@article{ye2024openfedllm:,
    title={OpenFedLLM: Training Large Language Models on Decentralized Private Data
  via Federated Learning},
    author={Rui Ye, Wenhao Wang, Jingyi Chai, Dihan Li, Zexi Li, Yinda Xu, Yaxin
  Du, Yanfeng Wang, Siheng Chen},
    journal={arXiv preprint arXiv:2402.06954},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.06954},
    primaryClass={cs.LG cs.CL cs.DC cs.MA}
}