BourGAN: Generative Networks with Metric Embeddings
This paper addresses the mode collapse for generative adversarial networks (GANs). We view modes as a geometric structure of data distribution in a metric space. Under this geometric lens, we embed subsamples of the dataset from an arbitrary metric space into the`2 space, while preserving their pairwise distance distribution. Not only does this metric embedding determine the dimensionality of the latent space automatically, it also enables us to construct a mixture of Gaussians to draw latent space random vectors. We use the Gaussian mixture model in tandem with a simple augmentation of the objective function to train GANs. Every major step of our method is supported by theoretical analysis, and our experiments on real and synthetic data confirm that the generator is able to produce samples spreading over most of the modes while avoiding unwanted samples, outperforming several recent GAN variants on a number of metrics and offering new features.
Review:   Mode collapse is a well-known problem in the GAN literature and it is an important one. The authors attribute the problem to the fact that GANs generate samples from a single Gaussian distribution, which has only one peak.  And they propose a fix to the problem by using a mixture of multiple Gaussian distributions.  In my view the diagnosis is clearly correct. It is so obvious that I am surprised that no one has thought of it before.  The solution proposed seems reasonable and has solid theoretical justification.  A key characteristic of this work is that it allows arbitrary user specified distance metric. The authors propose to use a classifier distance metric. It has been shown to improve generalization performance in the case of multiple modes as compared to the L2 distance and Earth Mover distance.  I found this very interesting.  The paper is also well-written and easy to follow.  I recommend that the paper be accepted.  While being very positive overall, I do have some concerns.  First, the number m of components in the Gaussian mixture needs to be provided by the user.  This is clearly a drawback. Moreover, I do not see any easy way to fix it as there isn’t any way to calculate model fit in GAN models. Second, the impact of m has not been sufficiently evaluated. This is an issue because a user is unlikely to get the correct value for m every time.    Other questions and comments:   - The proposed classifier distance is obtained from pre-trained classification networks. Does this mean that the propose method is a semi-supervised method rather than an unsupervised method?   - Why isn’t WGAN included in the comparisons?  - Please provide more information on the choices of hyper-parameters on Lines 200 and 204 and Line 512 in Supplementary Materials.  - The readability might be improved if the LPDD part is discussed before Wasserstein-1 distance in Section 3.1.  Confidence: The reviewer is confident but not absolutely certain about the proof of theorems.