Dual Discriminator Generative Adversarial Nets
We propose in this paper a novel approach to tackle the problem of mode collapse encountered in generative adversarial network (GAN). Our idea is intuitive but proven to be very effective, especially in addressing some key limitations of GAN. In essence, it combines the Kullback-Leibler (KL) and reverse KL divergences into a unified objective function, thus it exploits the complementary statistical properties from these divergences to effectively diversify the estimated density in capturing multi-modes. We term our method dual discriminator generative adversarial nets (D2GAN) which, unlike GAN, has two discriminators; and together with a generator, it also has the analogy of a minimax game, wherein a discriminator rewards high scores for samples from data distribution whilst another discriminator, conversely, favoring data from the generator, and the generator produces data to fool both two discriminators. We develop theoretical analysis to show that, given the maximal discriminators, optimizing the generator of D2GAN reduces to minimizing both KL and reverse KL divergences between data distribution and the distribution induced from the data generated by the generator, hence effectively avoiding the mode collapsing problem. We conduct extensive experiments on synthetic and real-world large-scale datasets (MNIST, CIFAR-10, STL-10, ImageNet), where we have made our best effort to compare our D2GAN with the latest state-of-the-art GAN's variants in comprehensive qualitative and quantitative evaluations. The experimental results demonstrate the competitive and superior performance of our approach in generating good quality and diverse samples over baselines, and the capability of our method to scale up to ImageNet database.
This paper presents a variant of generative adversarial networks (GANs) that utilizes two discriminators, one tries to assign high scores for data, and the other tries to assign high scores for the samples, both discriminating data from samples, and the generator tries to fool both discriminators.  It has been shown in section 3 that the proposed approach effectively optimizes the sum of KL and reverse KL between generator distribution and data distribution in the idealized non-parametric setup, therefore encouraging more mode coverage than other GAN variants.
The paper is quite well written and the formulation and analysis seems sound and straightforward.  The proposed approach is evaluated on toy 2D points dataset as well as more realistic MNIST, CIFAR-10, STL and ImageNet datasets.
I have one concern about the new formulation, as shown in Proposition 1, the optimal discriminators have the form of density ratios.  Would this cause instability issues?  The density ratios can vary wildly from 0 to infinity when the modes of the model and the data do not match well.  On the other hand, it is very hard for neural nets to learn to predict extremely large values with finite weights.  On the toy dataset, however, the proposed D2GAN seems to be more stable than standard GAN and unrolled GAN, is this in general true though?
I appreciate the authors’ effort in doing an extensive hyperparameter search for both the proposed method and the baselines on the MNIST dataset.  However, for a better comparison, it feels we should train the same generator network using all the methods compared, and tune the other hyperparameters for that network architecture (or a few networks).  Also, learning rate seems to be the most important hyperparameter but it is not searched extensively enough.
The paper claims they can scale up their proposed method to ImageNet, but I feel this is a little bit over-claiming, as they only tried on a subset of ImageNet and downsampled all images to 32x32, while there are already existing methods that can generate ImageNet samples for much higher resolution.
Overall the paper seems like a valid addition to the GAN literature, but doesn’t seem to change state-of-the-art too much compared to other approaches.