Despite the recent breakthroughs in unsupervised domain adaptation (uDA), no prior work has studied the challenges of applying these methods in practical machine learning scenarios. In this paper, we highlight two significant bottlenecks for uDA, namely excessive centralization and poor support for distributed domain datasets. Our proposed framework, MDDA, is powered by a novel collaborator selection algorithm and an effective distributed adversarial training method, and allows for uDA methods to work in a decentralized and privacy-preserving way.
###Summary###
This paper tackles unsupervised domain adaptation in a decentralized setting.  The high-level observation is that the conventional unsupervised domain adaptation has two bottlenecks, namely excessive centralization and poor support for distributed domain datasets. The paper proposes Multi-Step Decentralized Domain Adaptation (MDDA) to transfer the knowledge learned from the source domain to the target domain without sharing the data.  
The paper also explores explore a proposition: in addition to adapting from the labeled source, can uDA leverage the knowledge from other target domains, which themselves may have undergone domain adaptation in the past. 
The proposed MMDA method contains a feature extractor (E), a domain discriminator (D) and task classifier (C) for each domain. The target domain components are initialized with the respective source components. The source domain discriminator D_s target domain discriminator D_t are synchronized by exchanging and averaging the gradients. The paper also proposes Lazy Synchronization to reduce the communication cost of the algorithm.
The paper also proposes Wasserstein distance guided collaborator selection schema to perform the domain adaptation task. 
The paper performs experiments on five image and audio datasets: Rotated MNIST, Digits, and Office-Caltech, DomainNet and Mic2Mic. 
The baselines used in this paper include "Labeled Source", "Random Collaborator", and "Multi-Collaborator". The experimental results demonstrate that the proposed method can outperform the baselines on some of the experimental settings. The paper also provides a detailed analysis of the model and experimental results. 
### Novelty ###
This paper does not propose a new domain adaptation algorithm. However, the paper introduces some interesting tricks to solve the MMDA task such as the lazy synchronization between the source domain discriminator and the target domain discriminator. 
###Clarity###
Several critical explanations are missing from the paper:
1) When training the source domain discriminator D_s and target domain discriminator D_t, if the features between the source domain and target domain cannot be shared with each other, how to train the D_s and D_t. For example, the D_s cannot get access to the features from the target domain, how to train D_s? 
2) How is the target classifier C_t updated when there are no labels for the target domain?
3) As far as I understand, the domain discriminator is this paper is trained adversarially. The detailed adversarial training step is unclear.  
###Pros###
1) The paper proposes an interesting transfer learning schema where the data between the source and target domain can not be shared with each other to protect the data-privacy.
2) The paper provides extensive experiments on multiple standard domain adaptation benchmarks, especially the most recent dataset such as the DomainNet. 
3) The paper provides detail empirical analysis to demonstrate the effectiveness of the proposed methods. 
###Cons###
1) The most critical issue of this paper is that some explanations are missing, e.g. how are D_s, D_t, C_t trained? Refer to the #Clarity.
2) The presentation and writing of this paper need polish. The author should do more relative surveys to motivate the authors. One critical relevant reference of this paper is:
"Secure Federated Transfer Learning", Yang Liu et al 
https://arxiv.org/pdf/1812.03337.pdf
3) The baselines used in this paper is also trivial. It is desirable to compare the proposed method with state-of-the-art domain adaptation methods.
Based on the summary, cons, and pros, the current rating I am giving now is "reject". I would like to discuss the final rating with other reviewers, ACs.
To improve the rating, the author should explain the questions I proposed in the #Clarity