Exemplar Guided Unsupervised Image-to-Image Translation with Semantic Consistency
Image-to-image translation has recently received significant attention due to advances in deep learning. Most works focus on learning either a one-to-one mapping in an unsupervised way or a many-to-many mapping in a supervised way. However, a more practical setting is many-to-many mapping in an unsupervised way, which is harder due to the lack of supervision and the complex inner- and cross-domain variations. To alleviate these issues, we propose the Exemplar Guided & Semantically Consistent Image-to-image Translation (EGSC-IT) network which conditions the translation process on an exemplar image in the target domain. We assume that an image comprises of a content component which is shared across domains, and a style component specific to each domain. Under the guidance of an exemplar from the target domain we apply Adaptive Instance Normalization to the shared content component, which allows us to transfer the style information of the target domain to the source domain. To avoid semantic inconsistencies during translation that naturally appear due to the large inner- and cross-domain variations, we introduce the concept of feature masks that provide coarse semantic guidance without requiring the use of any semantic labels. Experimental results on various datasets show that EGSC-IT does not only translate the source image to diverse instances in the target domain, but also preserves the semantic consistency during the process.
 The paper is well organized with a clear idea of the proposed method and good related work descriptions. Overall, the descriptions are clear and easy to follow, but the experimental results need clarifying. - Regarding the multi-digit translation task, it is not straightforward to this reviewer how the proposed method could match the digits (semantic) with different colors (style) in different locations. The description in the paper is not enough to explain the results in Fig. 6. To this reviewer, this task is more complex than the street view translation one. In the same line, it is curious what the results would be if digits with different colors are overlapping at random location, rather than the grid-like arrangement. - For the potential readers who are not knowledgeable in semantic segmentation, please give the full name of mIoU for reference. - For further researches in this topic, it would be good to depict the limitations of the proposed method. For examples, the translated images in the CelebA dataset are not photorealistic (Fig. 8) and there are odd red lights in the middle of the results in GTA5<-BDD (Fig. 12). - typos: Fig. 2-caption: m_{a}->m_{A}
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Exemplar Guided Unsupervised Image-to-Image Translation with Semantic Consistency
Image-to-image translation has recently received significant attention due to advances in deep learning. Most works focus on learning either a one-to-one mapping in an unsupervised way or a many-to-many mapping in a supervised way. However, a more practical setting is many-to-many mapping in an unsupervised way, which is harder due to the lack of supervision and the complex inner- and cross-domain variations. To alleviate these issues, we propose the Exemplar Guided & Semantically Consistent Image-to-image Translation (EGSC-IT) network which conditions the translation process on an exemplar image in the target domain. We assume that an image comprises of a content component which is shared across domains, and a style component specific to each domain. Under the guidance of an exemplar from the target domain we apply Adaptive Instance Normalization to the shared content component, which allows us to transfer the style information of the target domain to the source domain. To avoid semantic inconsistencies during translation that naturally appear due to the large inner- and cross-domain variations, we introduce the concept of feature masks that provide coarse semantic guidance without requiring the use of any semantic labels. Experimental results on various datasets show that EGSC-IT does not only translate the source image to diverse instances in the target domain, but also preserves the semantic consistency during the process.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 The paper is well organized with a clear idea of the proposed method and good related work descriptions. Overall, the descriptions are clear and easy to follow, but the experimental results need clarifying. - Regarding the multi-digit translation task, it is not straightforward to this reviewer how the proposed method could match the digits (semantic) with different colors (style) in different locations. The description in the paper is not enough to explain the results in Fig. 6. To this reviewer, this task is more complex than the street view translation one. In the same line, it is curious what the results would be if digits with different colors are overlapping at random location, rather than the grid-like arrangement. - For the potential readers who are not knowledgeable in semantic segmentation, please give the full name of mIoU for reference. - For further researches in this topic, it would be good to depict the limitations of the proposed method. For examples, the translated images in the CelebA dataset are not photorealistic (Fig. 8) and there are odd red lights in the middle of the results in GTA5<-BDD (Fig. 12). - typos: Fig. 2-caption: m_{a}->m_{A}
assistant
 The paper is well organized with a clear idea of the proposed method and good related work descriptions. Overall, the descriptions are clear and easy to follow, but the experimental results need clarifying. - Regarding the multi-digit translation task, it is not straightforward to this reviewer how the proposed method could match the digits (semantic) with different colors (style) in different locations. The description in the paper is not enough to explain the results in Fig. 6. To this reviewer, this task is more complex than the street view translation one. In the same line, it is curious what the results would be if digits with different colors are overlapping at random location, rather than the grid-like arrangement. - For the potential readers who are not knowledgeable in semantic segmentation, please give the full name of mIoU for reference. - For further researches in this topic, it would be good to depict the limitations of the proposed method. For examples, the translated images in the CelebA dataset are not photorealistic (Fig. 8) and there are odd red lights in the middle of the results in GTA5<-BDD (Fig. 12). - typos: Fig. 2-caption: m_{a}->m_{A}