We consider the problem of domain generalization, namely, how to learn representations given data from a set of domains that generalize to data from a previously unseen domain. We propose the Domain Invariant Variational Autoencoder (DIVA), a generative model that tackles this problem by learning three independent latent subspaces, one for the domain, one for the class, and one for any residual variations. We highlight that due to the generative nature of our model we can also incorporate unlabeled data from known or previously unseen domains. To the best of our knowledge this has not been done before in a domain generalization setting. This property is highly desirable in fields like medical imaging where labeled data is scarce. We experimentally evaluate our model on the rotated MNIST benchmark and a malaria cell images dataset where we show that (i) the learned subspaces are indeed complementary to each other, (ii) we improve upon recent works on this task and (iii) incorporating unlabelled data can boost the performance even further.
In this work, the authors propose a domain invariant variational autoencoder for domain generalization problem. Specifically, the data is assumed to be constructed from three independent variables, one for the domain, one for the class and one for the residual variations. The method can be used for both unsupervised and semi-supervised cases. Experimental studies on rotated MNIST dataset and a malaria cell images dataset verify the effectiveness of the proposed method.
The paper is well-written and easy to follow. The proposed generative model is simple and technically sound. However, I have the following concerns.
(1)	It is not clear how the problem setting, i.e., domain generalization, matters in DIVA. In another words, the proposed DIVA is not specific to domain generalization problem, but can be used for domain adaptation, multiple source transfer learning etc. Actually, I find that the authors compare with DA, which is a conventional domain adaptation method, in the experiment. Moreover, the experimental setups in section 4.1.3 is a multi-source transfer setting, and DIVA can be well be applied. In this sense, I am not very convinced on the claim of the contribution that DIVA is proposed for domain generalization. For me, DIVA is a more general method.
(2)	With point 1, more related works on VAE for domain adaptation need to be discussed.
(3)	The idea of constructing data from disentangled latent variables is not new, see a latest work [ref1]. The main difference of DIVA from [ref1] is the residual variation variable. Two baselines are necessary for comparisons: (1) [ref1], and (2) DIVA without the residual variation variable. Actually, the semantic meaning of z_x is not well discussed. Even in the experimental studies figures 2 and 3, it is hard to tell what Z_x actually represents. 
(4)	Regarding the 4.1.2, why DA is selected as a baseline? How does DA deal with the multiple domains? Can any other domain adaptation methods, e.g., [ref2], or multiple source transfer methods, e.g. [ref3], be compared?
(5)	In the right side of Table 1, the improvements seem to be very marginal considering the variance. This makes the ability of DIVA to use unlabelled data less convincing. 
(6)	Regarding 4.1.3, it seems that the domain similarity plays an important role in the performance, comparing the results of M_{30} with M_{60}. Without the labelled M_{60}, which is very similar to the target M_{75}, the performance degenerates dramatically. The current DIVA treats all the domains equally, is it possible to have a weighted form of DIVA that distinguishes the contributions of different domains? 
(7)	What is the task of the malaria cell images experiments? Is it to classify the parasitized and uninfected cells? For a given patient, it makes more sense that all the cells belongs to one category, either infected or healthy. How is the class distribution for a person (in this case a domain)? Is it very unbalanced? 
(8)	For figure 3, It is hard to judge the cell images parasitized or uninfected without domain knowledge, can you give the label for each image? Again, the semantic meaning of Z-x is hard to tell. I am not convinced by the shape of the cell for Z-x.
(9)	For 4.2.2, why and how DA is compared? As far as I know, it is for unsupervised domain adaptation. Moreover, the improvements are quite marginal. 
Some minor comments:
(1)	Page1, first para, 3rd line, “present” -> “presented”.
(2)	Page2, first para, 2nd line, “Y” - > “Y denotes”
(3)	Some references lack of page information
The paper should be self-contained. I would suggest the authors move some paragraphs in appendix to the paper, for instance, 5.1.1, 5.2.2, and 5.2.3.
Overall, the paper is presented with extensive empirical evaluations, but less theoretical justification. The significance of the paper is moderate as the key idea of learning disentangled latent variables has been studied, and the paper lacks of evidence to show the pure benefits of introducing Z_x as well as the comparison with the related work [ref1]. 
[ref1] Learning Disentangled Semantic Representation for Domain Adaptation
[ref2] Conditional Adversarial Domain Adaptation
[ref3] Multiple Source Domain Adaptation with Adversarial Learning