Incorporating the desired invariance into representation learning is a key challenge in many situations, e.g., for domain generalization and privacy/fairness constraints. An adversarial invariance induction (AII) shows its power on this purpose, which maximizes the proxy of the conditional entropy between representations and attributes by adversarial training between an attribute discriminator and feature extractor (Xie et al., 2017). However, the practical behavior of AII is still unclear as the previous analysis assumes the optimality of the attribute classifier, which is rarely held in practice. This paper first analyzes the practical behavior of AII, indicating that AII has theoretical difficulty as it maximizes variational upper bound of the actual conditional entropy, and AII catastrophically fails to induce invariance even in simple cases as suggested by the above theoretical findings. We then argue that a simple modification to AII can significantly stabilize the adversarial induction framework and achieve better invariant representations. Our modification is based on the property of conditional entropy; it is maximized if and only if the divergence between all pairs of marginal distributions over representations between different attributes is minimized. The proposed method, invariance induction by discriminator matching, modify AII objective to explicitly consider the divergence minimization requirements. Empirical validations on both the toy dataset and four real-world datasets (related to applications of user anonymization and domain generalization) reveal that the proposed method provides superior performance when inducing invariance for nuisance factors.
*Summary*
The paper proposes a new method to learn data-driven representations, being invariant to some specific nuisance factors which are detrimental for the selected (supervised) classification task.
Authors build upon the existing probabilistic framework termed Adversarial Invariant Induction (AII) from (Xie et al., 2017). 
They claim to explore it under a both theoretical and practical point of view, demonstrating the limitations of maximizing a variational upper bound on conditional entropy as a proxy to achieve invariance. 
Leveraging these observation, authors propose a novel method, called “invariance induction by discriminator matching” (IIDM) that is based on a regularized classification loss, penalized by a Kullbach-Leibler divergence between conditional distributions of the nuisance factor.
Extremely convincing experiments are carried out on a synthetic and a real benchmark in multi-source domain generalization (PACS).
*Pros*
1. The genesis of the proposed IIDM is extremely paced since smoothly derived from the AII framework.
2. Experimental results on a synthetic benchmark (a version of rotated MNIST) and on a popular benchmark for domain generalization (PACS) proved the effectiveness of IIDM
 *Cons*
1. The paper is hard to get, if the reader is not familiar with related literature
2. It is not fully clear from the paper which parts are original and which are inherited from prior work.
3. The structure of the paper needs to be improved (check my comments in the section beneath)
Some of the proposed methodologies are not clear (IIDM+)

*Detailed Comments*
The problem considered by authors is surely interesting and addressing a popular topic in computer vision and deep learning. 
1. Unfortunately, the paper, as it is is hard to get for scholars which are not expert of the AII formalism, which, in my opinion is not enough detailed. Therefore, in my opinion clarity is something that authors should try to work hard on: for instance, during the rebuttal time, authors can write from scratch an entire new Section in which they explain in plain terms the main outcomes of their paper, without entering too much into technical details.
2. Additionally, the structure of the paper needs, in my opinion a major re-styling, still for the sake of better readability:
2.a. A visualization of the proposed method (for instance, using flow-diagrams) in comparison with the existing AII should help in rapidly getting the factors of novelty of the proposed IIDM method. I would also encourage authors to add a pseudo-code
2.b. Since authors claim two major contributions (understanding AII + IIDM), I would like to see those two contributions thoroughly presented and dissected in two separated sections of the paper. I am not fully convinced with the actual writing style in which the two contributions seem to be intertwined together.
3. Although already convincing, the experimental part can be improved:
3.a. Instead of a version of rotated-MNIST, authors can test on the “digits-five” setting (MNIST, MNIST-M, SVHN, UPS, SYN) as done in several works like http://openaccess.thecvf.com/content_cvpr_2018/papers/Xu_Deep_Cocktail_Network_CVPR_2018_paper.pdf.
3.b. In addition to multi-domain generalization, authors could also have tried more classical unsupervised domain adaptation settings or, even, single-source domain generalization as in https://papers.nips.cc/paper/7779-generalizing-to-unseen-domains-via-adversarial-data-augmentation.
*Final Evaluation*
I think that the main aspect that authors should face during the rebuttal is to make the paper more easy to read and better separate the two contributions (understanding AII and IIDM). What I am not convinced at all about the writing style of the authors since when reading the paper I am not always capable of understanding what is novel (since proposed by authors) and what is inherited from prior work. But, maybe, the reason for this is that I am not an expert of the specific related field - but, even so, I think that the paper needs to be understood from the broadest audience possible.
Instead, I am familiar with multi-source/single-source domain generalization (and adaptation) and, after my careful analysis of the experiments, I see a lot of potential in the approach. I would me more than interested in checking the performance of the proposed method over some of the novel benchmarks that I have recommended. It would be nice if authors add more experiments, but I know that this is always a complicated request during a rebuttal period.
Globally, if I were asked to only rate the experimental part, I would have promoted for full acceptance. Unfortunately, the theoretical part of the paper is not fully clear to me and, therefore, I am not confident in calling for a full acceptance only based on the experiments. 
In brief, I would go for a weak reject, looking forward to the authors’ rebuttal and the opinion of the other Fellow Reviewers.