Robust Training of Federated Models with Extremely Label Deficiency
<|reference_start|>Robust Training of Federated Models with Extremely Label Deficiency: Federated semi-supervised learning (FSSL) has emerged as a powerful paradigm for collaboratively training machine learning models using distributed data with label deficiency. Advanced FSSL methods predominantly focus on training a single model on each client. However, this approach could lead to a discrepancy between the objective functions of labeled and unlabeled data, resulting in gradient conflicts. To alleviate gradient conflict, we propose a novel twin-model paradigm, called Twin-sight, designed to enhance mutual guidance by providing insights from different perspectives of labeled and unlabeled data. In particular, Twin-sight concurrently trains a supervised model with a supervised objective function while training an unsupervised model using an unsupervised objective function. To enhance the synergy between these two models, Twin-sight introduces a neighbourhood-preserving constraint, which encourages the preservation of the neighbourhood relationship among data features extracted by both models. Our comprehensive experiments on four benchmark datasets provide substantial evidence that Twin-sight can significantly outperform state-of-the-art methods across various experimental settings, demonstrating the efficacy of the proposed Twin-sight.<|reference_end|>
@article{zhang2024robust,
    title={Robust Training of Federated Models with Extremely Label Deficiency},
    author={Yonggang Zhang, Zhiqin Yang, Xinmei Tian, Nannan Wang, Tongliang Liu,
  Bo Han},
    journal={arXiv preprint arXiv:2402.14430},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.14430},
    primaryClass={cs.LG}
}