EXEMPLAR GUIDED UNSUPERVISED IMAGE-TO- IMAGE TRANSLATION WITH SEMANTIC CONSISTENCY
Image-to-image translation has recently received significant attention due to advances in deep learning. Most works focus on learning either a one-to-one mapping in an unsupervised way or a many-to-many mapping in a supervised way. However, a more practical setting is many-to-many mapping in an unsupervised way, which is harder due to the lack of supervision and the complex inner-and cross-domain variations. To alleviate these issues, we propose the Exemplar Guided & Semantically Consistent Image-to-image Translation (EGSC-IT) network which conditions the translation process on an exemplar image in the target domain. We assume that an image comprises of a content component which is shared across domains, and a style component specific to each domain. Under the guidance of an exemplar from the target domain we apply Adaptive Instance Normalization to the shared content component, which allows us to transfer the style information of the target domain to the source domain. To avoid semantic inconsistencies during translation that naturally appear due to the large inner-and cross-domain variations, we introduce the concept of feature masks that provide coarse semantic guidance without requiring the use of any semantic labels. Experimental results on various datasets show that EGSC-IT does not only translate the source image to diverse instances in the target domain, but also preserves the semantic consistency during the process.
The introduction is written to perfection. The paper discusses a core failing and need for I2I translation models. The one-to-one mapping assumption does not apply to most tasks. While the approach seems novel the analysis of the results are insufficient to convince me that the method is really working. This should be a workshop paper.
For the motivation of the approach I am not convinced how the conditioned style is being used. It would be nice to see some analysis of how the latent space changes given different input images. Why would style information be propagated through the network? Why wouldn't noise work just as well? Although an abiliation study is performed there is no standard deviation reported so it is unclear if this number is fair. 
In Figure 5 the t-sne doesn't look correct. The points all seems to be projected on walls which could indicate some sort of overflow error. The text only devotes 3 lines to discuss this figure. It is not mentioned what part of the model the t-sne is computed from. To me this experiment that studies the internal representation is critical to convincing a reader to use this method. 
The segmentation results sound good. Where is the improvement coming from? The experimental section is cut short. The experiment section is really squeezed in the last two pages while the other sections are overly descriptive and could be reduced.
The figures should be changed to be visible without color (put a texture on each block).