Enhancing Data Quality in Federated Fine-Tuning of Foundation Models
<|reference_start|>Enhancing Data Quality in Federated Fine-Tuning of Foundation Models: In the current landscape of foundation model training, there is a significant reliance on public domain data, which is nearing exhaustion according to recent research. To further scale up, it is crucial to incorporate collaboration among multiple specialized and high-quality private domain data sources. However, the challenge of training models locally without sharing private data presents numerous obstacles in data quality control. To tackle this issue, we propose a data quality control pipeline for federated fine-tuning of foundation models. This pipeline computes scores reflecting the quality of training data and determines a global threshold for a unified standard, aiming for improved global performance. Our experiments show that the proposed quality control pipeline facilitates the effectiveness and reliability of the model training, leading to better performance.<|reference_end|>
@article{zhao2024enhancing,
    title={Enhancing Data Quality in Federated Fine-Tuning of Foundation Models},
    author={Wanru Zhao, Yaxin Du, Nicholas Donald Lane, Siheng Chen, Yanfeng Wang},
    journal={arXiv preprint arXiv:2403.04529},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.04529},
    primaryClass={cs.LG cs.AI cs.DC}
}