Unsupervised Neural Multi-Document Abstractive Summarization of Reviews
Abstractive summarization has been studied using neural sequence transduction methods with datasets of large, paired document-summary examples. However, such datasets are rare and the models trained from them do not generalize to other domains. Recently, some progress has been made in learning sequence-to-sequence mappings with only unpaired examples. In our work, we consider the setting where there are only documents (product or business reviews) with no summaries provided, and propose an end-to-end, neural model architecture to perform unsupervised abstractive summarization. Our proposed model consists of an auto-encoder trained so that the mean of the representations of the input reviews decodes to a reasonable summary-review. We consider variants of the proposed architecture and perform an ablation study to show the importance of specific components. We show through metrics and human evaluation that the generated summaries are highly abstractive, fluent, relevant, and representative of the average sentiment of the input reviews.
 # Positive aspects of this submission - This submission presents a really novel, creative, and useful way to achieve unsupervised abstractive multi-document summarization, which is quite an impressive feat. - The alternative metrics in the absence of ground-truth summaries seem really useful and can be reused for other summarization problems where ground-truth summaries are missing. In particular, the prediction of review/summary score as a summarization metric is very well thought of. - The model variations and experiments clearly demonstrate the usefulness of every aspect of the proposed model. # Criticism - The proposed model assumes that the output summary is similar in writing style and length to each of the inputs, which is not the case for most summarization tasks. This makes the proposed model hard to compare to the majority of previous works in supervised multi-document summarization like the ones evaluated on the DUC 2004 dataset. - The lack of applicability to existing supervised summarization use cases leaves unanswered the question of how much correlation there is between the proposed unsupervised metrics and existing metrics like the ROUGE score, even if they seem intuitively correlated. - This model suffers from the usual symptoms of other abstractive summarization models (fluency errors, factual inaccuracies). But this shouldn*t overshadow the bigger contributions of this paper, since dealing with these specific issues is still an open research problem.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Unsupervised Neural Multi-Document Abstractive Summarization of Reviews
Abstractive summarization has been studied using neural sequence transduction methods with datasets of large, paired document-summary examples. However, such datasets are rare and the models trained from them do not generalize to other domains. Recently, some progress has been made in learning sequence-to-sequence mappings with only unpaired examples. In our work, we consider the setting where there are only documents (product or business reviews) with no summaries provided, and propose an end-to-end, neural model architecture to perform unsupervised abstractive summarization. Our proposed model consists of an auto-encoder trained so that the mean of the representations of the input reviews decodes to a reasonable summary-review. We consider variants of the proposed architecture and perform an ablation study to show the importance of specific components. We show through metrics and human evaluation that the generated summaries are highly abstractive, fluent, relevant, and representative of the average sentiment of the input reviews.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 # Positive aspects of this submission - This submission presents a really novel, creative, and useful way to achieve unsupervised abstractive multi-document summarization, which is quite an impressive feat. - The alternative metrics in the absence of ground-truth summaries seem really useful and can be reused for other summarization problems where ground-truth summaries are missing. In particular, the prediction of review/summary score as a summarization metric is very well thought of. - The model variations and experiments clearly demonstrate the usefulness of every aspect of the proposed model. # Criticism - The proposed model assumes that the output summary is similar in writing style and length to each of the inputs, which is not the case for most summarization tasks. This makes the proposed model hard to compare to the majority of previous works in supervised multi-document summarization like the ones evaluated on the DUC 2004 dataset. - The lack of applicability to existing supervised summarization use cases leaves unanswered the question of how much correlation there is between the proposed unsupervised metrics and existing metrics like the ROUGE score, even if they seem intuitively correlated. - This model suffers from the usual symptoms of other abstractive summarization models (fluency errors, factual inaccuracies). But this shouldn*t overshadow the bigger contributions of this paper, since dealing with these specific issues is still an open research problem.
assistant
 # Positive aspects of this submission - This submission presents a really novel, creative, and useful way to achieve unsupervised abstractive multi-document summarization, which is quite an impressive feat. - The alternative metrics in the absence of ground-truth summaries seem really useful and can be reused for other summarization problems where ground-truth summaries are missing. In particular, the prediction of review/summary score as a summarization metric is very well thought of. - The model variations and experiments clearly demonstrate the usefulness of every aspect of the proposed model. # Criticism - The proposed model assumes that the output summary is similar in writing style and length to each of the inputs, which is not the case for most summarization tasks. This makes the proposed model hard to compare to the majority of previous works in supervised multi-document summarization like the ones evaluated on the DUC 2004 dataset. - The lack of applicability to existing supervised summarization use cases leaves unanswered the question of how much correlation there is between the proposed unsupervised metrics and existing metrics like the ROUGE score, even if they seem intuitively correlated. - This model suffers from the usual symptoms of other abstractive summarization models (fluency errors, factual inaccuracies). But this shouldn*t overshadow the bigger contributions of this paper, since dealing with these specific issues is still an open research problem.