INSTAGAN: INSTANCE-AWARE IMAGE-TO-IMAGE TRANSLATION
Unsupervised image-to-image translation has gained considerable attention due to the recent impressive progress based on generative adversarial networks (GANs). However, previous methods often fail in challenging cases, in particular, when an image has multiple target instances and a translation task involves significant changes in shape, e.g., translating pants to skirts in fashion images. To tackle the issues, we propose a novel method, coined instance-aware GAN (InstaGAN), that incorporates the instance information (e.g., object segmentation masks) and improves multi-instance transfiguration. The proposed method translates both an image and the corresponding set of instance attributes while maintaining the permutation invariance property of the instances. To this end, we introduce a context preserving loss that encourages the network to learn the identity function outside of target instances. We also propose a sequential mini-batch inference/training technique that handles multiple instances with a limited GPU memory and enhances the network to generalize better for multiple instances. Our comparative evaluation demonstrates the effectiveness of the proposed method on different image datasets, in particular, in the aforementioned challenging cases. Code and results are available in https://github.com/sangwoomo/instagan.
This paper does unpaired cross-domain translation of multi-instance images, proposing a method -- InstaGAN -- which builds on CycleGAN by taking into account instance information in the form of per-instance segmentation masks. 
=====================================
Pros:
The paper is well-written and easy to understand. The proposed method is novel, and does a good job of handling a type of information that previous methods couldn’t.
The motivation for each piece of the model and training objective is clearly explained in the context of the problem. Intuitively seems like a nice and elegant way to take advantage of the extra segmentation information available.
The results look pretty good and clearly compare favorably with CycleGAN and other baselines. The tested baselines seem like a fair comparison -- for example, the model capacity of the baseline is increased to compensate for the larger proposed model.
=====================================
Cons / suggestions:
The results are somewhat limited in terms of the number of domains tested -- three pairs of categories (giraffe/sheep, pants/skirts, cup/bottle).  In a sense, this is somewhat understandable -- one wouldn’t necessarily expect the method to be able to translate between objects with different scale or that are never seen in the same contexts (e.g. cups and giraffes). However, it would still have been nice to see e.g. more pairs of animal classes to confirm that the category pairs aren’t the only ones where the method worked.
Relatedly, it would have been interesting to see if a single model could be trained on multiple category pairs and benefit from information sharing between them.
The evaluation is primarily qualitative, with quantitative results limited to Appendix D showing a classification score. I think there could have been a few more interesting quantitative results, such as segmentation accuracy of the proposed images for the proposed masks, or reconstruction error. Visualizing some reconstruction pairs (i.e., x vs. Gyx(Gxy(x))) would have been interesting as well.
I would have liked to see a more thorough ablation of parts of the model. For example, the L_idt piece of the loss enforcing that an image in the target domain (Y) remain identical after passing through the generator mapping X->Y. This loss term could have been included in the original CycleGAN as well (i.e. there is nothing about it that’s specific to having instance information) but it was not -- is it necessary?
=====================================
Overall, while the evaluation could have been more thorough and quantitative, this is a well-written paper that proposes an interesting, well-motivated, and novel method with good results.
==========================================================================
REVISION
The authors' additional results and responses have addressed most of my concerns, and I've raised my rating from 6 to 7.
> We remark that the identity mapping loss L_idt is already used by the authors of the original CycleGAN (see Figure 9 of [2]). 
Thanks, you're right, I didn't know this was part of the original CycleGAN. As a final suggestion, it would be good to mention in your method section that this loss component is used in the original CycleGAN for less knowledgeable readers (like me) as it's somewhat hard to find in the original paper (only used in some of their experiments and not mentioned as part of the "main objective").