Federated learning improves data privacy and efficiency in machine learning performed over networks of distributed devices, such as mobile phones, IoT and wearable devices, etc. Yet models trained with federated learning can still fail to generalize to new devices due to the problem of domain shift. Domain shift occurs when the labeled data collected by source nodes statistically differs from the target node's unlabeled data. In this work, we present a principled approach to the problem of federated domain adaptation, which aims to align the representations learned among the different nodes with the data distribution of the target node. Our approach extends adversarial adaptation techniques to the constraints of the federated setting. In addition, we devise a dynamic attention mechanism and leverage feature disentanglement to enhance knowledge transfer. Empirically, we perform extensive experiments on several image and text classification tasks and show promising results under unsupervised federated domain adaptation setting.
The authors present a novel algorithm for dealing with domain adaptation in the setting of federated learning (classification, specifically). That is, they tackle the issue of learning a model on a new domain when access to the data points used in training the source models is not possible due to privacy constraints. The approach uses the gradients of the source models, reweighed to account for the differing shifts between the different sources and the target domain, to fit the model on the target domain.
The authors motivate their approach by providing a novel bound on the generalization error of transfer learning when the hypothesis function used on the target domain is a convex combination of hypotheses fitted on multiple source domains. This bound shows that the weighted sum of divergences in the symmetric difference hypothesis space controls the generalization error, so the authors aim at deriving feature representations and using aggregation weights that ensure this weighted sum is small. 
The authors use a novel dynamic attention model to get the aggregation weights: they cluster the features in the target domain, and measure how much the intra-cluster variation decreases when information from a given source domain is incorporated. The aggregation weights for the model updates on the target domain are then weighed using a softmax transform of these contribution weights. 
 
The motivation up through and including section 3 is clear, the theoretical results are presented clearly, but the model details in section 4 are unclear:
-  In the dynamic attention mechanism, how does one a priori choose the number of clusters in computing the gap statistics, and what is the impact? 
- the notation in the federated adversarial alignment section is unclear: what *exactly* are the model coefficients Theta that are being updated?
- the statement "optimize following objective" is made several times. this is ambiguous, and should be corrected to "miminize" following objective.
- the representation disentanglement process is intricate, and only vaguely addressed. how does one fit the neural net and use (8)? where is the l2 reconstruction loss balanced with the mutual information? the vagueness of this section means Algorithm 1 is not well-specified.
The experiments are reasonable, and compare to baseline domain adaptation methods.
The problem considered is of interest, and the approach is novel and interesting. However, the algorithm is not described in sufficient detail. After reading the paper, and spending considerable time rereading section 4, I still do not understand how Algorithm 1 is implemented in practice. For that reason I lean towards reject. I will update my score if the authors clarify the details of Algorithm 1. 
Comments:
- the symmetric difference hypothesis space is incorrectly called the HdeltaH divergence in section 3