EMERGING DISENTANGLEMENT IN AUTO-ENCODER BASED UNSUPERVISED IMAGE CONTENT TRANSFER
We study the problem of learning to map, in an unsupervised way, between domains A and B, such that the samples b ∈ B contain all the information that exists in samples a ∈ A and some additional information. For example, ignoring occlusions, B can be people with glasses, A people without, and the glasses, would be the added information. When mapping a sample a from the first domain to the other domain, the missing information is replicated from an independent reference sample b ∈ B. Thus, in the above example, we can create, for every person without glasses a version with the glasses observed in any face image. Our solution employs a single two-pathway encoder and a single decoder for both domains. The common part of the two domains and the separate part are encoded as two vectors, and the separate part is fixed at zero for domain A. The loss terms are minimal and involve reconstruction losses for the two domains and a domain confusion term. Our analysis shows that under mild assumptions, this architecture, which is much simpler than the literature guided-translation methods, is enough to ensure disentanglement between the two domains. We present convincing results in a few visual domains, such as no-glasses to glasses, adding facial hair based on a reference image, etc.
This paper tackles the task of content transfer. For a given type of images (frontal face shots), the goal is to transfer a particular localized property (e.g. glasses or facial hair) extracted from one image to another image of the same type (difference face). This is also known as the problem of guided image-to-image translation. 
The problem is formalized as the one of learning to map two different domains, one domain being composed of images with the property/attribute of interest, the other one containing images without it. The problem is said to be ‘unsupervised’, i.e. there is no pairwise correspondences between images of the two domains (with/without attributes).
The novelty of the approach lies on
-	the loss, which is composed of three terms: two reconstruction losses and a domain confusion loss
-	the overall architecture and in particular the fact that images are represented as a combination of the output of two encoders: one encodes the face and the other encodes the property (e.g. glasses).
Overall comments:
+ a theoretical part discusses generalization bounds and the emergence of disentangled representations
+ visual results are appealing showing the suitability of the method to the considered task
- the discussion of the advantages of the proposed method could be improved
- the motivation for some of the experimental results is unclear (choice of experimental protocol and baselines). 
- the scope of the method seems limited
Detailed comments:
I personally like the described model. The disentanglement mechanism is intuitive to understand, and seems well suited for this particular task, as qualitative evidence suggests. I am not sure if this approach would be applicable beyond the very specific scenario considered in the paper. 
The paper emphasizes that the strength of the method lies on its simplicity w.r.t. competitors, and its better results. These two aspects could be better discussed. 
Simplicity: 
In several places the paper claims that the proposed approach is considerably simpler. Some parts hint to criteria for the ‘complexity’ comparison, such as Table 1 or a few sentences (e.g. “this allows us to train with many less parameters and without the need to applying excessive tuning”). It would be more convincing to have a dedicated discussion of the practical advantages of the simplicity claimed by this method, discussing e.g. training/testing time, memory footprint of the models, convergence properties, stability, etc. 
Comparison: 
The chosen baselines, i.e. MUNIT and DRIT are experimentally shown to perform poorly on the considered task. Yet although these methods were also developed for guided image translation, they were designed for a rather different application: style transfer. I am not sure these comparisons bring much insight on the performance of the method.
Experiments are conducted for a very specific task, on a single dataset. Would the method have broader application?
Experimental protocol:
I understand that such an approach is difficult to evaluate quantitatively but I am not sure what there is to learn from experiments reported in Table 3, as there is no point of comparison on this task. This could be clarified. 
Additional comments:
-	The paper relies on the assumption that the distribution of persons with sunglasses and that of persons without them is the same, except for the sunglasses. This sounds like a strong requirement for the data used to train the network; it would be interesting to discuss the practical impact of this assumption, especially on the data requirement for the method to perform well
-	I found Figure 1 quite useful. A visual representation of the architecture and its associated description help follow the technical part. 
-	I got confused with some of the claims in section 4.2. More generally, I found the technical part hard to follow.
-	The user study seems small: only 10 pairs of images are considered. How were those pair chosen? Is the set representative?