DIVERSITY-SENSITIVE CONDITIONAL GENERATIVE ADVERSARIAL NETWORKS
We propose a simple yet highly effective method that addresses the mode-collapse problem in the Conditional Generative Adversarial Network (cGAN). Although conditional distributions are multi-modal (i.e., having many modes) in practice, most cGAN approaches tend to learn an overly simplified distribution where an input is always mapped to a single output regardless of variations in latent code. To address such issue, we propose to explicitly regularize the generator to produce diverse outputs depending on latent codes. The proposed regularization is simple, general, and can be easily integrated into most conditional GAN objectives. Additionally, explicit regularization on generator allows our method to control a balance between visual quality and diversity. We demonstrate the effectiveness of our method on three conditional generation tasks: image-to-image translation, image inpainting, and future video prediction. We show that simple addition of our regularization to existing models leads to surprisingly diverse generations, substantially outperforming the previous approaches for multi-modal conditional generation specifically designed in each individual task.
The paper proposes a simple way of addressing the issue of mode-collapse by adding a regularisation to force the outputs to be diverse. Specifically, a loss is added that maximises the l2 loss between the images generated, normalised by the distance between the corresponding latent codes. This method is also used to control the balance between visual quality and diversity.
The paper is overall well written, introducing and referencing well existing concepts, and respected the 8 pages recommendation.
Why was the maximum theta (the bound for numerical stability) incorporated in equation 2? What happens if this is omitted in practice? How is this determined?
In section 4, an increase of the gradient norm of the generator is implied: does this have any effect on the robustness/sensitivity of the model to adversarial attacks?
In section 5, how is the “appropriate CGAN” determined?
My main issue is with the experimental setting that is somewhat lacking. The visual quality of the samples illustrated in the paper is inferior to that observed in the state-of-the-art, begging the question of whether this is a tradeoff necessary to obtain better diversity or if it is a consequence of the additional regularisation.. The diversity observed seems to mainly be attributable to colour differences rather than more elaborate differences. Even quantitatively, the proposed method seems only marginally better than other methods.
Update post rebuttal
-----------------------------
The experimental setting that is a little lacking. Qualitatively and quantitatively, the improvements seem marginal, with no significant improvement shown. I would have liked a better study of the tradeoff between visual quality and diversity, if necessary at all.
However, the authors addressed well the issues. Overall, the idea is interesting and simple and, while the paper could be improved with some more work, it would benefit the ICLR readership in its current form, so I would recommend it as a poster -- I am increasing my score to that effect.