A DEEP REINFORCED MODEL FOR ABSTRACTIVE SUMMARIZATION
Attentional, RNN-based encoder-decoder models for abstractive summarization have achieved good performance on short input and output sequences. For longer documents and summaries however these models often include repetitive and incoherent phrases. We introduce a neural network model with a novel intraattention that attends over the input and continuously generated output separately, and a new training method that combines standard supervised word prediction and reinforcement learning (RL). Models trained only with supervised learning often exhibit "exposure bias" -they assume ground truth is provided at each step during training. However, when standard word prediction is combined with the global sequence prediction training of RL the resulting summaries become more readable. We evaluate this model on the CNN/Daily Mail and New York Times datasets. Our model obtains a 41.16 ROUGE-1 score on the CNN/Daily Mail dataset, an improvement over previous state-of-the-art models. Human evaluation also shows that our model produces higher quality summaries.
The paper proposes a model for abstractive document summarization using a self-critical policy gradient training algorithm, which is mixed with maximum likelihood objective. The Seq2seq architecture incorporates both intra-temporal and intra-decoder attention, and a pointer copying mechanism. A hard constraint is imposed during decoding to avoid trigram repetition. Most of the modelling ideas already exists, but this paper show how they can be applied as a strong summarization model.
The approach obtains strong results on the CNN/Daily Mail and NYT datasets. Results show that intra-attention improves performance for only one of the datasets. RL results are reported with only the best-performing attention setup for each dataset. My concern with that is that the authors might be using the test set for model selection; It is not a priori clear that the setup that works better for ML should also be better for RL, especially as it is not the same across datasets. So I suggest that results for RL should be reported with and without intra-attention on both datasets, at least on the validation set.
It is shown that intra-decoder attention decoder improves performance on longer sentences. It would be interesting to see more analysis on this, especially analyzing what the mechanism is attending to, as it is less clear what its interpretation should be than for intra-temporal attention. Further ablations such as the effect of the trigram repetition constraint will also help to analyse the contribution of different modelling choices to the performance. 
For the mixed decoding objective, how is the mixing weight chosen and what is its effect on performance? If it is purely a scaling factor, how is the scale quantified? It is claimed that readability correlates with perplexity, so it would be interesting to see perplexity results for the models. The lack of correlation between automatic and human evaluation raises interesting questions about the evaluation of abstractive summarization that should be investigated further in future work.
This is a strong paper that presents a significant improvement in document summarization.