Adversarial Ranking for Language Generation
Generative adversarial networks (GANs) have great successes on synthesizing data. However, the existing GANs restrict the discriminator to be a binary classifier, and thus limit their learning capacity for tasks that need to synthesize output with rich structures such as natural language descriptions. In this paper, we propose a novel generative adversarial network, RankGAN, for generating high-quality language descriptions. Rather than training the discriminator to learn and assign absolute binary predicate for individual data sample, the proposed RankGAN is able to analyze and rank a collection of human-written and machine-written sentences by giving a reference group. By viewing a set of data samples collectively and evaluating their quality through relative ranking scores, the discriminator is able to make better assessment which in turn helps to learn a better generator. The proposed RankGAN is optimized through the policy gradient technique. Experimental results on multiple public datasets clearly demonstrate the effectiveness of the proposed approach.
The paper proposes an alternative parameterization of the GAN discriminator based on relative ranks among a comparison set. The method shows better quantitative results for language generation on three (relatively small) language datasets.
Mathematically, there are some potential problems of the proposed method. 
- Firstly, the original GAN identified the equilibrium of Eqn. (1) only for R(s|U,C) being a proper probability over a Bernoulli variable. However, the definition in Eqn. (4) does not guarantee this condition. Thus, it is difficult to see whether the proposed criterion can ensure the generator distribution to match the true data distribution. 
- However, if one regards the expected log probability in Eqn. (4) as a definition of energy, the practically used criterion Eqn. (7) is more similar to the energy-based GAN. From this perspective, if one completes the two expectations on U and C omitted in Eqn. (7), (i.e., the full formula for the a term should be E_{s ~ P_h} [ E_{U ~ bar{U}} E_{C ~ bar{C}} log R(s|U,C) ] where bar{U} and bar{C} represent all possible reference and comparison sets respectively), and plugs Eqn. (4) into the full formula, it becomes E_{s ~ P_h} [ E_{U ~ bar{U}} E_{C ~ bar{C}} ( E_{u ~ U} log P(s|u,C) ) ] <= E_{s ~ P_h} [ log P(s) ], which is a lower bound of the ranker-defined likelihood. In other words, the energy function is defined as a lower bound of the likelihood, which is a little bit strange and not conventional. 
- Putting things together, I don’t really understand what the proposed criterion is doing, and why it leads to better empirical performances. 
In terms of the experiment result, despite the good quantitative results, it would be nice if some qualitative analysis were conducted to understand why and how the generated sentences could achieve higher scores. Also, is the result sensitive to the size of the comparison set |C| or the size of the reference set |U|? 
In summary, the proposed method achieves nice empirical results in improving language generation under the GAN setting. However, given the content of the paper, it is still not clear why this method should lead to such improvement.