Generative Adversarial Networks (GANs) are known to be difficult to train, despite considerable research effort. Several regularization techniques for stabilizing training have been proposed, but they introduce non-trivial computational overheads and interact poorly with existing techniques like spectral normalization. In this work, we propose a simple, effective training stabilizer based on the notion of consistency regularization-a popular technique in the semi-supervised learning literature. In particular, we augment data passing into the GAN discriminator and penalize the sensitivity of the discriminator to these augmentations. We conduct a series of experiments to demonstrate that consistency regularization works effectively with spectral normalization and various GAN architectures, loss functions and optimizer settings. Our method achieves the best FID scores for unconditional image generation compared to other regularization methods on CIFAR-10 and CelebA. Moreover, Our consistency regularized GAN (CR-GAN) improves stateof-the-art FID scores for conditional generation from 14.73 to 11.48 on CIFAR-10 and from 8.73 to 6.66 on ImageNet-2012.
This paper proposes to use Consistency Regularization for training GANs, a technique known to work well in unsupervised learning. The technique consists in applying a transformation to real images and enforcing that the features of the discriminator between the transformed inputs and the original inputs are similar. The author show that using this technique enables them to improve the performance of a standard GAN significantly on CIFAR10. They also carry an ablation study studying the influence of the different part of the proposed technique.
Overall I'm in favor of accepting this paper. The paper is well written, with convincing experiments and an interesting ablation study. However I have several minor issues that I think could greatly improve the paper if addressed.
Minor comments:
- I think an idea which is somewhat related but hasn't been mentioned in the paper, is the idea of adding noise to the input when training GANs [1]. I think this is worth mentioning in the related work.
- Related to the previous point, why penalizing features and not directly output ? What about also trying to classify the transformed images as real ? Also you say that penalizing the last layer, I think including the influence of m (eq 2) in the ablation study would be interesting.
- The authors provide some measure of standard deviation on some experiments but not on all of them. It would be nice to systematically report the standard deviation for every experiments.
- In figure 1 the author make the hypothesis that the discriminator will output very different score to images semantically close together. Did the author verify this hypothesis experimentally ?
- Also why penalizing only the samples from the real distribution and not from the generator ? have you tried both ?
- When the test accuracy of the discriminator is low, it could also be that the discriminator is under-fitting, it would be nice to also report the train accuracy for the discriminator.
- I think the conclusion about the effect of consistency regularization vs data augmentation is a bit vague since consistency regularization has no sense without data-augmentation. 
- It's quite interesting but also disappointing that combining transformations doesn't give that much of an improvement. Do the author have any intuition why this is the case ? and why learning them one after the other would work ?
References:
[1] Arjovsky and Bottou. "Towards Principled Methods for Training Generative Adversarial Networks." (ICLR 2017)