IMPROVING GENERALIZATION AND STABILITY OF GENERATIVE ADVERSARIAL NETWORKS
Generative Adversarial Networks (GANs) are one of the most popular tools for learning complex high dimensional distributions. However, generalization properties of GANs have not been well understood.
In this paper, we analyze the generalization of GANs in practical settings. We show that discriminators trained on discrete datasets with the original GAN loss have poor generalization capability and do not approximate the theoretically optimal discriminator. We propose a zero-centered gradient penalty for improving the generalization of the discriminator by pushing it toward the optimal discriminator. The penalty guarantees the generalization and convergence of GANs. Experiments on synthetic and large scale datasets verify our theoretical analysis.
The primary innovation of this paper seems focused towards increasing the generalization of GANs, while also maintaining convergence and preventing mode collapse.
The authors first discuss common pitfalls concerning the generalization capability of discriminators, providing analytical underpinnings for their later experimental results. Specifically, they address the problem of gradient explosion in discriminators. 
The authors then suggest that a zero-centered gradient penalty (0-GP) can be helpful in addressing this issue. 0-GPs are regularly used in GANs, but the authors point out that the purpose is usually to  provide convergence, not to increase generalizability. Non-zero centered penalties can give a convergence guarantee but, the authors, assert, can allow overfitting. A 0-GP can give the same guarantees but without allowing overfitting to occur.
The authors then verify these assertions through experimentation on synthetic data, as well as MNIST and ImageNet. My only issue here is that very little information was given about the size of the training sets. Did they use all the samples? Some portion? It is not clear from reading. This would be a serious impediment to reproducibility.
All in all, however, the authors provide a convincing  combination of analysis and experimentation. I believe this paper should be accepted into ICLR.
Note: there is an error on page 9, in Figure 3. The paragraph explanation should list that the authors' 0-GP is figure 3(e). They list (d) twice.