We propose Cooperative Training (CoT) for training generative models that measure a tractable density for discrete data. CoT coordinately trains a generator G and an auxiliary predictive mediator M . The training target of M is to estimate a mixture density of the learned distribution G and the target distribution P , and that of G is to minimize the Jensen-Shannon divergence estimated through M . CoT achieves independent success without the necessity of pre-training via Maximum Likelihood Estimation or involving high-variance algorithms like REINFORCE. This low-variance algorithm is theoretically proved to be superior for both sample generation and likelihood prediction. We also theoretically and empirically show the superiority of CoT over most previous algorithms in terms of generative quality and diversity, predictive generalization ability and computational cost.
The paper proposes an interesting method, where the discriminator is replaced by a component that estimates the density that is the mixture of the data and the generator's distributions. In a sense, that component is only a device that allows estimating a Jensen-Shannon divergence for the generator to then be optimized against. Other GAN papers have replaced their discriminator by a similar device (e.g., WGANs, ..), but the present formulation seems novel. The numerical experiments presented on a synthetic Turing test and text generation from EMNLP's 2017 news dataset appear promising. 
Overall, the mediator seems to allow to achieve lower Jensen-Shannon (JS) divergence values in the experiments (and is kind of designed for that). Although this may be an improvement with respect to existing methods for discrete sequential data, it may also be limited in that it may not easily extend to other types of divergences that have proved superior to JS in some continuous settings.
The paper is rather clear, although there are lots of small grammatical errors as well as odd formulations which end up being distracting or confusing. The language should be proof-read carefully. 
Pros:
- Generative modeling of sequence data still in its infancy
- Potentially lower variance than policy gradient approaches
- Experiments are promising
Cons:
- Lots of grammatical errors and odd formulations
Questions:
- Equation 14: what does it mean to find the "maximum entropy solution" for the given optimization problem?
- Figure 2: how do (b) and (c) relate to each other?
Remarks, small typos and odd formulations:
- "for measuring M_\/phi": what does measuring mean in this context?
- What does small m refer to? Algorithm 1 says the total number of steps  but it is also used in the main text as an index for J and \pi (for mediator?)
- Equation block 8: J_m has not been defined yet
- "the supports of distributions G and P"... -> G without subscript has now been defined in this context
- "if the training being perfect"
- "tend to get stuck in some sub-optimals"
- the learned distribution "collapseS"
- "since  the data distribution is, thus ..."
- "that measures a" -> "that estimates a ..."?
- "a predictive module": a bit unclear - generative v. discriminative is more usual terminology
- "is well ensured"
- "with the cost of diversity" -> "at the cost of diversity"?
- "has theoretical guarantee"
- in the references: "ALIAS PARTH GOYAL" (all caps)
- "let p denote the intermediate states": I don't understand what this is. Where is "p" used? (proof of Theorem 3)
- "CoT theoretically guarantees the training effectiveness": what does that mean?
- Figure 3: "epochs" -> "Epochs"
- Algorithm 1: what does "mixed balanced samples" mean? Make this more precise
- "wide-ranged"
- Equation 10 is too long and equation number is not properly formatted
- Figures hard to read in black & white
- Figure 2 doesn't use the same limits for the Y axis of the two NLL plots, making comparisons difficult. The two NLL plots are also not side-by-side