Unsupervised domain adaptation aims to generalize the hypothesis trained in a source domain to an unlabeled target domain. One popular approach to this problem is to learn domain-invariant embeddings for both domains. In this work, we study, theoretically and empirically, the effect of the embedding complexity on generalization to the target domain. In particular, this complexity affects an upper bound on the target risk; this is reflected in experiments, too. Next, we specify our theoretical framework to multilayer neural networks. As a result, we develop a strategy that mitigates sensitivity to the embedding complexity, and empirically achieves performance on par with or better than the best layer-dependent complexity tradeoff.
This paper studies the impact of embedding complexity on domain-invariant representations. By incorporating embedding complexity into the previous upper bound explicitly, the authors demonstrate the limitations of previous theories and algorithms. Based on their theoretical findings, the authors propose to control the embedding complexity with implicit regularization. Specifically, aligning source and target feature distributions in multiple layers controls both embedding complexity and domain discrepancy. The proposed algorithm can achieve similar performance as DANN with manual selection of embedding depth.
By noting that the hypothesis space can be decomposed in to the feature extractor and the classifier, the authors propose to address the domain discrepancy separately. D_H\DeltaH is termed latent divergence, which the algorithm attempts to minimize. D_G\DeltaG is treated as embedding complexity, which is the intrinsic property of the feature extractor. Thus, domain-invariant representations should seek a proper tradeoff between those two terms. 
The paper is well-written and the contributions are stated clearly. The exploration on the layer division is really insightful.
However, I have several concerns: 
1.	The proposed upper bound is insightful, but it has several limitations. Compared to the version applied to the feature space in equation (3), the proposed upper bound is looser. The embedding complexity terms includes two encoders, which are deep neural networks in practice, thus it can be excessively large. As the authors point out, in equation (3), the embedding complexity is not addressed explicitly, but it is implicit in the adaptability \lambda in a more reasonable way. Previous works [1], [2], [3] have already taken them into consideration. Proposition 5 is a direct application of proposition 1 in [1].
2.	On the claim of implicit regularization. By applying domain adversarial training to multiple layers, the authors claim that the encoder in higher layers is implicitly restricted. However, they do not validate this regularization effect. Is the embedding complexity controlled? Theoretical analysis or experimental results would be helpful.
3.	The proposed MDM method seems to be incremental. [4] has probed into the effect of multi-layer adaptation strategy. Besides, applying domain adversarial training to many layers leads to more computational cost and may slow down training significantly. 
[1]Fredrik D Johansson, Rajesh Ranganath, and David Sontag. Support and invertibility in domain- invariant representations. arXiv preprint arXiv:1903.03448, 2019.
[2]Han Zhao, Remi Tachet des Combes, Kun Zhang, and Geoffrey J Gordon. On learning invariant representation for domain adaptation. arXiv preprint arXiv:1901.09453, 2019.
[3] Hong Liu, Mingsheng Long, Jianmin Wang, and Michael Jordan. Transferable adversarial training: A general approach to adapting deep classifiers. In International Conference on Machine Learning, pp. 4013–4022, 2019.
[4] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I. Jordan. Learning transferable features with deep adaptation networks. In Proceedings of the 32nd International Conference on International Conference on Machine Learning, volume 37, pp. 97–105, 2015.