Published as a conference paper at ICLR 2018 CGANS WITH PROJECTION DISCRIMINATOR
We propose a novel, projection based way to incorporate the conditional information into the discriminator of GANs that respects the role of the conditional information in the underlining probabilistic model. This approach is in contrast with most frameworks of conditional GANs used in application today, which use the conditional information by concatenating the (embedded) conditional vector to the feature vectors. With this modification, we were able to significantly improve the quality of the class conditional image generation on ILSVRC2012 (ImageNet) 1000-class image dataset from the current state-of-the-art result, and we achieved this with a single pair of a discriminator and a generator. We were also able to extend the application to super-resolution and succeeded in producing highly discriminative super-resolution images. This new structure also enabled high quality category transformation based on parametric functional transformation of conditional batch normalization layers in the generator. The code with Chainer (Tokui et al., 2015), generated images and pretrained models are available at https://github.com/pfnet-research/sngan_projection.
I thank the authors for the thoughtful response and updated manuscript. After reading through both, my review score remains unchanged.
=================
The authors describe a new variant of a generative adversarial network (GAN) for generating images. This model employs a 'projection discriminator' in order to incorporate image labels and demonstrate that the resulting model outperforms state-of-the-art GAN models.
Major comments:
1) Spatial resolution. What spatial resolution is the model generating images at? The AC-GAN work performed an analysis to assess how information is being introduced at each spatial resolution by assessing the gains in the Inception score versus naively resizing the image. It is not clear how much the gains of this model is due to generating better lower resolution images and performing simple upscaling. It would be great to see the authors address this issue in a serious manner.
2) FID in real data. The numbers in Table 1 appear favorable to the projection model. Please add error bars (based on Figure 4, I would imagine they are quite large). Additionally, would it be possible to compute this statistic for *real* images? I would be curious to know what the FID looks like as a 'gold standard'.
3) Conditional batch normalization.  I am not clear how much of the gains arose from employing conditional batch normalization versus the proposed method for incorporating the projection based discriminator. The former has been seen to be quite powerful in accomodating multi-modal tasks (e.g. https://arxiv.org/abs/1709.07871, https://arxiv.org/abs/1610.07629
). If the authors could provide some evidence highlighting the marginal gains of one technique, that would be extremely helpful.
Minor comments:
- I believe you have the incorrect reference for conditional batch normalization on Page 5.
A Learned Representation For Artistic Style
Dumoulin, Shlens and Kudlur (2017)
https://arxiv.org/abs/1610.07629
- Please enlarge images in Figure 5-8. Hard to see the detail of 128x128 images.
- Please add citations for Figures 1a-1b. Do these correspond with some known models?
Depending on how the authors respond to the reviews, I would consider upgrading the score of my review.