Generative Adversarial Networks (GANs) are powerful tools for realistic image generation. However, a major drawback of GANs is that they are especially hard to train, often requiring large amounts of data and long training time. In this paper we propose the Deli-Fisher GAN, a GAN that generates photo-realistic images by enforcing structure on the latent generative space using similar approaches by DeliGAN as introduced in Gurumurthy et al. (2017). The structure of the latent space we consider in this paper is modeled as a mixture of Gaussians, whose parameters are learned in the training process. Furthermore, to improve stability and efficiency, we use the Fisher Integral Probability Metric as the divergence measure in our GAN model, instead of the Jensen-Shannon divergence. We show by experiments that the Deli-Fisher GAN performs better than DC-GAN, WGAN, and the Fisher GAN as measured by inception score.
- Summary
This paper presents a minor improvement over the previous Deli-GAN framework (Gurumurthy et al CVPR'17). Specifically, the work proposes to use the Fisher Integral Probability Metric as the divergence measure in our GAN model, instead of the Jensen-Shannon divergence. It shows little results (seems to be positive) using this new distance measure other than traditional ones.  Except that, I didn't see any other contribution from this paper.
- Suggestions
The paper is poorly written and seems to be a rush submission to ICLR. For example:
* a lot of grammatical errors throughout the paper
* only 6.5 papes out of 8 pages are utilized
* the introduction is not convincing -- what problem are you going to address? any summary of your methodology? why it is expected to outperform existing frameworks? what distinguishes your work from existing works? and what're your main results? I cannot conclude after reading the intro.
* the results are very minor and not convincing. It seems the authors conducted a very limited set of experiments and concluded that the proposed Deli-Fisher GAN is better. If you claim that the proposed framework can generate better images, at least the framework should be compared to the latest state-of-the-art GANs (e.g. spectral GANs, etc.)
* The writing is not polished. 
Overall, the paper is far from ready to be submitted to ICLR, not mentioning acceptance. I would recommend the authors to conduct more experiments and comparisons and do a better job before submitting it to future conferences.