We study how, in generative adversarial networks, variance in the discriminator's output affects the generator's ability to learn the data distribution. In particular, we contrast the results from various well-known techniques for training GANs when the discriminator is near-optimal and updated multiple times per update to the generator. As an alternative, we propose an additional method to train GANs by explicitly modeling the discriminator's output as a bi-modal Gaussian distribution over the real/fake indicator variables. In order to do this, we train the Gaussian classifier to match the target bi-modal distribution implicitly through metaadversarial training. We observe that our new method, when trained together with a strong discriminator, provides meaningful, non-vanishing gradients.
The authors provided empirical analysis of different variants of GANs and proposed a regularization scheme to combat the vanishing gradient when the discriminator is well trained. 
More specifically, the authors demonstrated the importance of intra-class variance in the discriminator’s output. Methods whose discriminators tend to map inputs of a class to single real values are unable to provide a reliable learning signal for the generator, such as the standard GAN and Least Squares GAN. Variance in the discriminator’s output is essential to allow the generator to learn in the presence of a well-trained discriminator. To ensure the discriminator’s output follows the mixture of two univariate Gaussians, the authors proposed to add two additional discriminators which are trained in a similar was as the original GAN formulation. The technique is related to Linear Discriminant Analysis. From a broader perspective, the new meta-adversarial learning can be applied to ensure various desirable properties in GANs.
The performance of variance regularization scheme was evaluated on the CIFAR-10 and CelebA data.
Summary:
——
I think the paper discusses a very interesting topic and presents an interesting direction for training the GANs. A few points are missing which would provide significantly more value to readers. See comments below for details and other points.
Comments:
——
1.	Why would a bi-modal distribution be meaningful? Deep nets implicitly transform the data which is probably much more effective than using complex bi-modal Gaussian distribution; the bi-modal concept can likely be captured using classical techniques.
2.	On page 4, in Eq. (8) and (9), it remains unclear what $\mathcal{R}$ and $\mathcal{F}$ really are beyond two-layer MLPs; are the results of those two-layer MLPs used as the mean of a Gaussian distribution, i.e., $\mu_r$ and $\mu_f$?
3.	Regarding the description above Eq. (12), what is really used in practice, i.e., in the experiments? The paper omits many details that seem important for understanding. Could the authors provide more details on choosing the generator loss function and why Eq. (12) provides satisfying results in practice?    
Minor Comments:
——
1.	In Sec 2.1, the sentence needs to be corrected: “As shown in Arjovsky & Bottou (2017), the JS divergence will be flat everywhere important if P and Q both lie on low-dimensional manifolds (as is likely the case with real data) and do not prefectly align.”
2.	Last sentence in Conclusion: “which can be applied to ensure enforce various desirable properties in GANs.” Please remove either “ensure” or “enforce.”