Published as a conference paper at ICLR 2018 BOUNDARY-SEEKING GENERATIVE ADVERSARIAL NETWORKS
Generative adversarial networks (GANs, Goodfellow et al., 2014) are a learning framework that rely on training a discriminator to estimate a measure of difference between a target and generated distributions. GANs, as normally formulated, rely on the generated samples being completely differentiable w.r.t. the generative parameters, and thus do not work for discrete data. We introduce a method for training GANs with discrete data that uses the estimated difference measure from the discriminator to compute importance weights for generated samples, thus providing a policy gradient for training the generator. The importance weights have a strong connection to the decision boundary of the discriminator, and we call our method boundary-seeking GANs (BGANs). We demonstrate the effectiveness of the proposed algorithm with discrete image and character-based natural language generation. In addition, the boundary-seeking objective extends to continuous data, which can be used to improve stability of training, and we demonstrate this on Celeba, Large-scale Scene Understanding (LSUN) bedrooms, and Imagenet without conditioning.
Thanks for the feedback and for clarifying the 1) algorithm and the assumptions in the multivariate case 2) comparison to RL based methods 3) connection to estimating importance sampling weights using GAN discriminator.
I think the paper contribution is now more clear and strengthened with additional convincing experiments and I am increasing my score to 7.
The paper would still benefit from doing the experiment with importance weights by pixel , rather then a global one as done in the paper now. I encourage the authors to still do the experiment, see if there is any benefit.
==== Original Review =====
Summary of the paper:
The paper presents a method based on importance sampling and reinforcement learning to learn discrete generators in the GAN framework. The GAN uses an  f-divergence cost function for  training the discriminator. The generator is trained to minimize the KL distance between the  discrete generator q_{\theta}(x|z), and the importance weight discrete real distribution estimator w(x|z)q(\theta|z). where w(x|z) is estimated in turn using the discriminator. 
The methodology is also extended to the continuous case. Experiments are conducted on quantized image generation, and text generation.
Quality:
the paper is overall well written and supported with reasonable experiments.
Clarity:
The paper has a lot of typos that make sometimes the paper harder to follow:
- page (2) Eq 3 max , min should be min, max if we want to keep working with f-divergence
- Definition 2.1 \mathbb{Q}_{\theta} --> \mathbb{Q}
- page 5 the definition of \tilde{w}(x^{(m})) in the normalization it is missing \tilde{w}
- Equation (10) \nabla_{\theta}\log(x|z) --> \nabla_{\theta}\log(x^{(m)}|z)
- In algorithm 1, again missing indices in the update of theta  --> \nabla_{\theta}\log(x^{(m|n)}|z^{n})
 
Originality:
The main ingredients of the paper are well known and already used in the literature (Reinforce for discrete GAN with Disc as a reward for e.g GAN for image captioning Dai et al). The perspective from importance sampling coming from f-divergence for discrete GAN has some novelty although the foundations of this work relate also to previous work:
- Estimating ratios using the discriminator is well known for e.g learning implicit models , Mohamed et al 
- The relation of  importance sampling to  reinforce is also well known" On a Connection between Importance Sampling and the Likelihood Ratio Policy Gradient," Tang and Abbeel.
General Review:
- when the generator is producing only *one* discrete distribution the theory is presented in Section 2.3. When we move to experiments, for image generation for example, we need to have a generator that produces a distribution by pixel. It would be important for 1) understanding the work 2) the reproducibility of the work to parallel algorithm 1 and have it *in the paper*, for this 'multi discrete distribution ' generation case.  If we have N pixels    \log(p(x_1,...x_N|z))= \Pi_i g_{\theta}(x_i|z) (this should be mentioned in the paper if it is the case ), it would be instructive to comment on the assumptions on independence/conditional dependence of this model, also to state clearly how the generator is updated in this case and what are importance sampling weights. 
- Would it make sense in this N pixel discrete case generation to have also the discriminator produce N probabilities of real and fake as in PixelGAN in Isola et al? then see in this case what are the importance sampling weights this would parallel the instantaneous reward in RL?