Synergizing Foundation Models and Federated Learning: A Survey
<|reference_start|>Synergizing Foundation Models and Federated Learning: A Survey: The recent development of Foundation Models (FMs), represented by large language models, vision transformers, and multimodal models, has been making a significant impact on both academia and industry. Compared with small-scale models, FMs have a much stronger demand for high-volume data during the pre-training phase. Although general FMs can be pre-trained on data collected from open sources such as the Internet, domain-specific FMs need proprietary data, posing a practical challenge regarding the amount of data available due to privacy concerns. Federated Learning (FL) is a collaborative learning paradigm that breaks the barrier of data availability from different participants. Therefore, it provides a promising solution to customize and adapt FMs to a wide range of domain-specific tasks using distributed datasets whilst preserving privacy. This survey paper discusses the potentials and challenges of synergizing FL and FMs and summarizes core techniques, future directions, and applications. A periodically updated paper collection on FM-FL is available at https://github.com/lishenghui/awesome-fm-fl.<|reference_end|>
@article{li2024synergizing,
    title={Synergizing Foundation Models and Federated Learning: A Survey},
    author={Shenghui Li, Fanghua Ye, Meng Fang, Jiaxu Zhao, Yun-Hin Chan, Edith
  C.-H. Ngai, Thiemo Voigt},
    journal={arXiv preprint arXiv:2406.12844},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.12844},
    primaryClass={cs.LG cs.AI}
}