Published as a conference paper at ICLR 2018 COULOMB GANS: PROVABLY OPTIMAL NASH EQUI- LIBRIA VIA POTENTIAL FIELDS
Generative adversarial networks (GANs) evolved into one of the most successful unsupervised techniques for generating realistic images. Even though it has recently been shown that GAN training converges, GAN models often end up in local Nash equilibria that are associated with mode collapse or otherwise fail to model the target distribution. We introduce Coulomb GANs, which pose the GAN learning problem as a potential field, where generated samples are attracted to training set samples but repel each other. The discriminator learns a potential field while the generator decreases the energy by moving its samples along the vector (force) field determined by the gradient of the potential field. Through decreasing the energy, the GAN model learns to generate samples according to the whole target distribution and does not only cover some of its modes. We prove that Coulomb GANs possess only one Nash equilibrium which is optimal in the sense that the model distribution equals the target distribution. We show the efficacy of Coulomb GANs on LSUN bedrooms, CelebA faces, CIFAR-10 and the Google Billion Word text generation.
In this paper, the authors interpret the training of GAN by potential field and inspired from which to provide new training procedure for GAN. They claim that under the condition that global optima are achieved for discriminator and generator in each iteration, the Coulomb GAN converges to the global solution. 
I think there are several points need to be addressed. 
1, I agree that the "model collapsing" is due to converging to a local Nash Equilibrium. However, there are more reasons besides the drawback of the loss function, which is emphasized in the paper. Leave the stochastic gradient descent optimization algorithm apart (since most of the neural networks are trained in this way), the parametrization and the richness of discriminator family play a vital role in the model collapsing issue. In fact, even with KL-divergence in which log operation is involved, if one can select reasonable parametrization, e.g., directly handling in function space, the saddle point optimization is convex-concave, which means under the same assumption made in the paper, there is only one global Nash Equilibrium. On the other hand, the richness of the discriminator also important in the training of GAN. I did not get the point about the drawback of III. If indeed as the paper considered in the ideal case, the discriminator is rich enough, III cannot happen. 
The model collapsing is not just because loss function in training GAN. It is caused by the twist of these three issues listed above. Modifying the loss can avoid partially model collapsing, however, it is not appropriate to claim that the proposed algorithm is 'provable'. The assumption in this paper is too restricted, and the discussion is unfair to the existing variants of GAN, e.g., GMMN or Wasserstein GAN, which under some assumptions, there is also only one global Nash Equilibrium. 
2, In the training procedure, the discriminator family is important as we discussed. The paper claims that the reason to introduce the extra discriminator is reducing variance. However, such parametrization will introduce bias too. The bias and variance tradeoff should be explicitly discussed here. Ideally, it should contain all the functions formed with Plummer kernel, but not too large (otherwise, it will increase the sample complexity.). Which function family used in the paper is not clear. 
3, As the authors already realized, the GMMN is one closely related model. It will be more convincing to add the comparison with GMMN. 
In sum, this paper provides an interesting perspective modeling GAN from the potential field, however, there are several issues need to be addressed. I expect to see the reply of the authors regarding the mentioned issues.