Hierarchical Summary-to-Article Generation
In this paper, we explore 	extit{summary-to-article generation}: the task of generating long articles given a short summary, which provides finer-grained content control for the generated text. To prevent sequence-to-sequence (seq2seq) models from degenerating into language models and better controlling the long text to be generated, we propose a hierarchical generation approach which first generates a sketch of intermediate length based on the summary and then completes the article by enriching the generated sketch. To mitigate the discrepancy between the ``oracle** sketch used during training and the noisy sketch generated during inference, we propose an end-to-end joint training framework based on multi-agent reinforcement learning. For evaluation, we use text summarization corpora by reversing their inputs and outputs, and introduce a novel evaluation method that employs a summarization system to summarize the generated article and test its match with the original input summary. Experiments show that our proposed hierarchical generation approach can generate a coherent and relevant article based on the given summary, yielding significant improvements upon conventional seq2seq models.
This paper explores the task of summary-to-article generation, the task of generating long articles given a short summary. Generating long text as output is challenging, since the seq2seq model often fall into degeneration of language models. To address this issue and have a better control on the long-form text generation, this paper proposes a hierarchical generation approach which first generates an intermediate sketch of the article and then generated the full article. This intermediate sketch can be noisy during inference because of the discrepancy between the training and inference strategies. To this end, this paper proposes a multi-agent reinforcement learning as well. For the evaluation of such long-form text outputs (article), this paper also proposes to use a summarization model to summarize the generated article and measure the ROUGE score between the generated summary and ground-truth. The premise here is that if we can generate a good article, then the output summary generated based on this article as input will also be good. They empirically evaluated the proposed hierarchical model with reinforcement learning, showing significant improvements over conventional seq2seq models. Overall, this paper is interesting with some new ideas, with caveat for some clarifications and missing some important experiments. Arguments: 1) It would be interesting to see how does a fine-tuned GPT-2 model on these summarization datasets perform on this article generation task. 2) ROUGE-rec metric is weird in the sense that if the summary-to-article model just copies the summary with some junk non-related information which is fluent, the summarization model based on this article can still generate a good summary. However, this might not be the case as the human evaluation correlation is good with this metric. Anyway, I would suggest to further analyse this metric. 3) By seeing the outputs generated by the conventional seq2seq model, I was wondering if the repetition can be fixed by simply blocking the n-gram repeats. Further, I would be interesting to see if transformers are better baselines in such long-form text generation tasks.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Hierarchical Summary-to-Article Generation
In this paper, we explore 	extit{summary-to-article generation}: the task of generating long articles given a short summary, which provides finer-grained content control for the generated text. To prevent sequence-to-sequence (seq2seq) models from degenerating into language models and better controlling the long text to be generated, we propose a hierarchical generation approach which first generates a sketch of intermediate length based on the summary and then completes the article by enriching the generated sketch. To mitigate the discrepancy between the ``oracle** sketch used during training and the noisy sketch generated during inference, we propose an end-to-end joint training framework based on multi-agent reinforcement learning. For evaluation, we use text summarization corpora by reversing their inputs and outputs, and introduce a novel evaluation method that employs a summarization system to summarize the generated article and test its match with the original input summary. Experiments show that our proposed hierarchical generation approach can generate a coherent and relevant article based on the given summary, yielding significant improvements upon conventional seq2seq models.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper explores the task of summary-to-article generation, the task of generating long articles given a short summary. Generating long text as output is challenging, since the seq2seq model often fall into degeneration of language models. To address this issue and have a better control on the long-form text generation, this paper proposes a hierarchical generation approach which first generates an intermediate sketch of the article and then generated the full article. This intermediate sketch can be noisy during inference because of the discrepancy between the training and inference strategies. To this end, this paper proposes a multi-agent reinforcement learning as well. For the evaluation of such long-form text outputs (article), this paper also proposes to use a summarization model to summarize the generated article and measure the ROUGE score between the generated summary and ground-truth. The premise here is that if we can generate a good article, then the output summary generated based on this article as input will also be good. They empirically evaluated the proposed hierarchical model with reinforcement learning, showing significant improvements over conventional seq2seq models. Overall, this paper is interesting with some new ideas, with caveat for some clarifications and missing some important experiments. Arguments: 1) It would be interesting to see how does a fine-tuned GPT-2 model on these summarization datasets perform on this article generation task. 2) ROUGE-rec metric is weird in the sense that if the summary-to-article model just copies the summary with some junk non-related information which is fluent, the summarization model based on this article can still generate a good summary. However, this might not be the case as the human evaluation correlation is good with this metric. Anyway, I would suggest to further analyse this metric. 3) By seeing the outputs generated by the conventional seq2seq model, I was wondering if the repetition can be fixed by simply blocking the n-gram repeats. Further, I would be interesting to see if transformers are better baselines in such long-form text generation tasks.
assistant
This paper explores the task of summary-to-article generation, the task of generating long articles given a short summary. Generating long text as output is challenging, since the seq2seq model often fall into degeneration of language models. To address this issue and have a better control on the long-form text generation, this paper proposes a hierarchical generation approach which first generates an intermediate sketch of the article and then generated the full article. This intermediate sketch can be noisy during inference because of the discrepancy between the training and inference strategies. To this end, this paper proposes a multi-agent reinforcement learning as well. For the evaluation of such long-form text outputs (article), this paper also proposes to use a summarization model to summarize the generated article and measure the ROUGE score between the generated summary and ground-truth. The premise here is that if we can generate a good article, then the output summary generated based on this article as input will also be good. They empirically evaluated the proposed hierarchical model with reinforcement learning, showing significant improvements over conventional seq2seq models. Overall, this paper is interesting with some new ideas, with caveat for some clarifications and missing some important experiments. Arguments: 1) It would be interesting to see how does a fine-tuned GPT-2 model on these summarization datasets perform on this article generation task. 2) ROUGE-rec metric is weird in the sense that if the summary-to-article model just copies the summary with some junk non-related information which is fluent, the summarization model based on this article can still generate a good summary. However, this might not be the case as the human evaluation correlation is good with this metric. Anyway, I would suggest to further analyse this metric. 3) By seeing the outputs generated by the conventional seq2seq model, I was wondering if the repetition can be fixed by simply blocking the n-gram repeats. Further, I would be interesting to see if transformers are better baselines in such long-form text generation tasks.