PROBGAN: TOWARDS PROBABILISTIC GAN WITH THEORETICAL GUARANTEES
Probabilistic modelling is a principled framework to perform model aggregation, which has been a primary mechanism to combat mode collapse in the context of Generative Adversarial Networks (GAN). In this paper, we propose a novel probabilistic framework for GANs, ProbGAN, which iteratively learns a distribution over generators with a carefully crafted prior. Learning is efficiently triggered by a tailored stochastic gradient Hamiltonian Monte Carlo with a novel gradient approximation to perform Bayesian inference. Our theoretical analysis further reveals that our treatment is the first probabilistic framework that yields an equilibrium where generator distributions are faithful to the data distribution. Empirical evidence on synthetic high-dimensional multi-modal data and image databases (CIFAR-10, STL-10, and ImageNet) demonstrates the superiority of our method over both start-of-the-art multi-generator GANs and other probabilistic treatment for GANs.
Summary
=========
The paper extends Bayesian GANs by altering the generator and discriminator parameter likelihood distributions and their respective priors. 
The authors further propose an SGHMC algorithm to collect samples of the resulting posterior distributions on each parameter set and evaluate their approach on both a synthetic and the CIFAR-10 data set. 
They claim superiority of their method, reporting a higher distance to mode centers of generated data points and better generator space coverage for the synthetic data set and better inception scores for the real world data for their method.
Review
=========
As an overall comment, I found the language poor, at times misleading.
The authors should have their manuscript proof-read for grammar and vocabulary.
Examples: 
- amazing superiority (page 1, 3rd paragraph)
- Accutally... (page 1, end of 3rd paragraph)
- the total mixture of generated data distribution (page 3, mid of 3.1)
- Similarity we define (page 3, end of 3.1)
- etc.
Over the whole manuscript, determiners are missing.
The authors start out with a general introduction to GANs and Bayesian GANs in particular, 
arguing that it is an open research question whether the generator converges to the true data generating distribution in Bayesian GANs.
I do not agree here. The Bayesian GAN defines a posterior distribution for the generator that
is proportional to the likelihood that the discriminator assigns to generated samples.
The better the generator, the higher the likelihood that the discriminator assign to these samples.
In the case of a perfect generator, here the discriminator is equally unable to distinguish real and generated samples and consequently degenerates to a constant function.
Using the same symmetry argument as the authors, one can show that this is the case for Bayesian GANs.
While defining the likelihood functions, the iterator variable t is used without introduction.
Further, I a confused by their argument of incompatibility.
First, they derive a Gibbs style update scheme based on single samples for generator and discriminator parameters using
posteriors in which the noise has been explicitly marginalized out by utilizing a Monte Carlo estimate.
Second, the used posteriors are conditional distributions with non-identical conditioning sets.
I doubt that the argument still holds under this setting.
With respect to the remaining difference between the proposed approach and Bayesian GAN,
I'd like the authors elaborate where exactly the difference between expectation of objective value
and objective value of expectation is.
Since the original GAN objectives used for crafting the likelihoods are deterministic functions,
randomness is introduced by the distributions over the generator and discriminator parameters.
I would have guessed that expectations propagate into the objective functions.
It is, however, interesting to analyze the proposed inference algorithm, especially the introduced posterior distributions.
For the discriminator, this correspond simply to the likelihood function.
For the generator, the likelihood is combined with some prior for which no closed form solution exists.
In fact, this prior changes between iterations of the inference algorithm.
The resulting gradient of the posterior decomposes into the gradient of the current objective and the sum over all previous gradients.
While this is not a prior in the Bayesian sense (i.e. in the sense of an actual prior belief), it would be interesting to have a closer look at the effect this has on the sampling method.
My educated guess is, that this conceptually adds up to the momentum term in SGHMC and thus slows down the exploration of the parameter space and results in better coverage.
The experiments are inspired by the ones done in the original Bayesian GAN publication.
I liked the developed method to measure coverage of the generator space although I find the
term of hit error misleading.
Given that the probabilistic methods all achieve a hit rate of 1, a lower hit error actually points to worse coverage.
I was surprised to see that hit error and coverage are only not consistently negatively correlated.
Adding statistics over several runs of the models (e.g. 10) would strengthen the claim of superior performance.