  Learning from limited data has been extensively studied in machine learning,
considering that deep neural networks achieve optimal performance when trained
using a large amount of samples. Although various strategies have been proposed
for centralized training, the topic of federated learning with small datasets
remains largely unexplored. Moreover, in realistic scenarios, such as settings
where medical institutions are involved, the number of participating clients is
also constrained. In this work, we propose a novel federated learning
framework, named RepTreeFL. At the core of the solution is the concept of a
replica, where we replicate each participating client by copying its model
architecture and perturbing its local data distribution. Our approach enables
learning from limited data and a small number of clients by aggregating a
larger number of models with diverse data distributions. Furthermore, we
leverage the hierarchical structure of the client network (both original and
virtual), alongside the model diversity across replicas, and introduce a
diversity-based tree aggregation, where replicas are combined in a tree-like
manner and the aggregation weights are dynamically updated based on the model
discrepancy. We evaluated our method on two tasks and two types of data, graph
generation and image classification (binary and multi-class), with both
homogeneous and heterogeneous model architectures. Experimental results
demonstrate the effectiveness and outperformance of RepTreeFL in settings where
both data and clients are limited. Our code is available at
https://github.com/basiralab/RepTreeFL.

Machine Learning