Published as a conference paper at ICLR 2018 GENERATING WIKIPEDIA BY SUMMARIZING LONG SEQUENCES
We show that generating English Wikipedia articles can be approached as a multidocument summarization of source documents. We use extractive summarization to coarsely identify salient information and a neural abstractive model to generate the article. For the abstractive model, we introduce a decoder-only architecture that can scalably attend to very long sequences, much longer than typical encoderdecoder architectures used in sequence transduction. We show that this model can generate fluent, coherent multi-sentence paragraphs and even whole Wikipedia articles. When given reference documents, we show it can extract relevant factual information as reflected in perplexity, ROUGE scores and human evaluations.
The main significance of this paper is to propose the task of generating the lead section of Wikipedia articles by viewing it as a multi-document summarization problem. Linked articles as well as the results of an external web search query are used as input documents, from which the Wikipedia lead section must be generated. Further preprocessing of the input articles is required, using simple heuristics to extract the most relevant sections to feed to a neural abstractive summarizer. A number of variants of attention mechanisms are compared, including the transofer-decoder, and a variant with memory-compressed attention in order to handle longer sequences. The outputs are evaluated by ROUGE-L and test perplexity. There is also a A-B testing setup by human evaluators to show that ROUGE-L rankings correspond to human preferences of systems, at least for large ROUGE differences.
This paper is quite original and clearly written. The main strength is in the task setup with the dataset and the proposed input sources for generating Wikipedia articles. The main weakness is that I would have liked to see more analysis and comparisons in the evaluation.
Evaluation:
Currently, only neural abstractive methods are compared. I would have liked to see the ROUGE performance of some current unsupervised multi-document extractive summarization methods, as well as some simple multi-document selection algorithms such as SumBasic. Do redundancy cues which work for multi-document news summarization still work for this task?
Extractiveness analysis:
I would also have liked to see more analysis of how extractive the Wikipedia articles actually are, as well as how extractive the system outputs are. Does higher extractiveness correspond to higher or lower system ROUGE scores? This would help us understand the difficulty of the problem, and how much abstractive methods could be expected to help. 
A further analysis which would be nice to do (though I have less clear ideas how to do it), would be to have some way to figure out which article types or which section types are amenable to this setup, and which are not. 
I have some concern that extraction could do very well if you happen to find a related article in another website which contains encyclopedia-like or definition-like entries (e.g., Baidu, Wiktionary) which is not caught by clone detection. In this case, the problem could become less interesting, as no real analysis is required to do well here.
Overall, I quite like this line of work, but I think the paper would be a lot stronger and more convincing with some additional work.
----
After reading the authors' response and the updated submission, I am satisfied that my concerns above have been adequately addressed in the new version of the paper. This is a very nice contribution.