  We consider a real-world scenario in which a newly-established pilot project
needs to make inferences for newly-collected data with the help of other
parties under privacy protection policies. Current federated learning (FL)
paradigms are devoted to solving the data heterogeneity problem without
considering the to-be-inferred data. We propose a novel learning paradigm named
transductive federated learning (TFL) to simultaneously consider the structural
information of the to-be-inferred data. On the one hand, the server could use
the pre-available test samples to refine the aggregated models for robust model
fusion, which tackles the data heterogeneity problem in FL. On the other hand,
the refinery process incorporates test samples into training and could generate
better predictions in a transductive manner. We propose several techniques
including stabilized teachers, rectified distillation, and clustered label
refinery to facilitate the model refinery process. Abundant experimental
studies verify the superiorities of the proposed \underline{M}odel
\underline{r}efinery framework for \underline{T}ransductive
\underline{F}ederated learning (MrTF). The source code is available at
\url{https://github.com/lxcnju/MrTF}.

Machine Learning