RELGAN: RELATIONAL GENERATIVE ADVERSARIAL NETWORKS FOR TEXT GENERATION
Generative adversarial networks (GANs) have achieved great success at generating realistic images. However, the text generation still remains a challenging task for modern GAN architectures. In this work, we propose RelGAN, a new GAN architecture for text generation, consisting of three main components: a relational memory based generator for the long-distance dependency modeling, the Gumbel-Softmax relaxation for training GANs on discrete data, and multiple embedded representations in the discriminator to provide a more informative signal for the generator updates. Our experiments show that RelGAN outperforms current state-of-the-art models in terms of sample quality and diversity, and we also reveal via ablation studies that each component of RelGAN contributes critically to its performance improvements. Moreover, a key advantage of our method, that distinguishes it from other GANs, is the ability to control the trade-off between sample quality and diversity via the use of a single adjustable parameter. Finally, RelGAN is the first architecture that makes GANs with Gumbel-Softmax relaxation succeed in generating realistic text.
==========================
I have read the authors' response and other reviewers' comments. Thanks the authors for taking great effort in answering my questions. Generally, I feel satisfied with the repsonse, and prefer an acceptance recommendation. 
==========================
Contributions:
The main contribution of this paper is the proposed RelGAN. First, instead of using a standard LSTM as generator, the authors propose using a relational memory based generator. Second, instead of using a single CNN as discriminator, the authors use multiple embedded representations. Third, Gumbel-softmax relaxation is also used for training GANs on discrete textual data. The authors also claim the proposed model has the ability to control the trade-off between sample quality and diversity via a single adjustable parameter. 
Detailed comments:
(1) Novelty: This paper is not a breakthrough paper, mainly following previous work and propose new designs to improve the performance. However, it still contains some novelty inside, for example, the model choice of the generator and discriminator. I think the observation that the temperature control used in the Gumbel-softmax can reflect the trade-off between quality and diversity is interesting. 
However, I feel the claim in the last sentence of the abstract and introduction is a little bit strong. Though this paper seems to be the first to really use Gumbel-softmax for text generation, similar techniques like using annealed softmax to approximate argmax has already been used in previous work (Zhang et al., 2017). Since this is similar to Gumbel-softmax, I think this may need additional one or two sentences to clarify this for more careful discussion.  
Further, I would also recommend the authors discuss the following paper [a] to make this work more comprehensive as to the discussion of related work. [a] also uses annealed softmax approximation, and also divide the GAN approaches as RL-based and RL-free, similar in spirit as the discuss in this paper. 
[a] Adversarial Text Generation via Feature-Mover's Distance, NIPS 2018.
(2) Presentation: This paper is carefully written and easy to follow. I enjoyed reading the paper. 
(3) Evaluation: Experiments are generally well-executed, with ablation study also provided. However, human evaluation is lacked, which I think is essential for this line of work. I have a few questions listed below. 
Questions:
(1) In section 2.4, it mentions that the generator needs pre-training. So, my question is: does the discriminator also need pre-training? If so, how the discriminator is pre-trained?
(2) In Table 1 & 2 & 3, how does your model compare with MaskGAN? If this can be provided, it would be better. 
(3) Instead of using NLL_{gen}, a natural question is: what are the self-BLEU score results since it was used in previous work?
(4) The \beta_max value used in the synthetic and real datasets is quite different. For example, \beta_max = 1 or 2 in synthetic data, while \beta_max = 100 or 1000 is used in real data. What is the observation here? Can the authors provide some insights into this?
(5) I feel Figure 3 is interesting. As the authors noted, NLL_gen measures diversity, NLL_oracle measures quality. Looking at Figure 3, does this mean GAN model produces higher quality samples than MLE pretrained models, while GAN models also produces less diverse samples than MLE models? This is due to NLL_gen increases after pretraining, while NLL_oracle further decreases after pretraining. However, this conclusion also seems strange. Can the authors provide some discussion on this? 
(6) Can human evaluation be performed since automatic metrics are not reliable enough?