Toward Controllable Text Content Manipulation
Controlled generation of text is of high practical use. Recent efforts have made impressive progress in generating or editing sentences with given textual attributes (e.g., sentiment). This work studies a new practical setting of text content manipulation. Given a structured record, such as (PLAYER: Lebron, POINTS: 20, ASSISTS: 10), and a reference sentence, such as Kobe easily dropped 30 points, we aim to generate a sentence that accurately describes the full content in the record, with the same writing style (e.g., wording, transitions) of the reference. The problem combines the characteristics of data-to-text generation and style transfer, and is challenging to minimally yet effectively manipulate the text (by rewriting/adding/deleting text portions) to ensure fidelity to the structured content. We derive two datasets from the data-to-text task as our testbed, and develop a neural method with weakly supervised competing objectives and explicit content coverage constraints. Automatic and human evaluations show superiority of our approach over competitive methods including a strong rule-based baseline and prior approaches designed for style transfer.
This paper addresses an interesting case of text generation, interpolating between the requirements of content fidelity and style transfer. This is carried out using a hybrid *attention-copy* mechanism, which is intended to attend to style in a reference sentence and copy content over from a structured data representation. This along with data augmentation and a combined unsupervised objective comprise the learning framework. An ablation study and human evaluation are carried out, confirming that the model presented has advantages over foregoing style transfer and generation models. I think this paper should be accepted as it addresses an interesting and widely applicable scenario, and the combined unsupervised objective is a creative and well-motivated solution. It consitutes a contribution both to neural generation and style transfer methods. It is not a strong accept as I think that when introducing a new task, more in-depth evaluation and error analysis is appropriate. One challenge with this task is evaluation -- some of the scores are close enough to 50% that it would be useful to have significance of some kind calculated, more information about the annotators and inter annotator agreement, and some amount of error analysis to help give context for Table 3. Particularly, since this is a new task, the foregoing models compared would be expected to perform worse than the new model. To learn more about ways in which the new model is well adapted to the task, it could be useful to look closer into the tradeoff between fidelity and style transfer, e.g. comapring the BLEU score between y and y_hat in addition to y* and y_hat for all models. Recommendations: 1. If possible, please use clearer notation to explain the data augmentation step. It is really interesting and creative, but very hard to follow x, x*, y, y* -- please consider using subscripts like y_side or something. 2. In section 6.2, a binary transformer classifier is mentioned. Please clarify how *manipulating the content successfully* is measured, and describe the classification task in further detail.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Toward Controllable Text Content Manipulation
Controlled generation of text is of high practical use. Recent efforts have made impressive progress in generating or editing sentences with given textual attributes (e.g., sentiment). This work studies a new practical setting of text content manipulation. Given a structured record, such as (PLAYER: Lebron, POINTS: 20, ASSISTS: 10), and a reference sentence, such as Kobe easily dropped 30 points, we aim to generate a sentence that accurately describes the full content in the record, with the same writing style (e.g., wording, transitions) of the reference. The problem combines the characteristics of data-to-text generation and style transfer, and is challenging to minimally yet effectively manipulate the text (by rewriting/adding/deleting text portions) to ensure fidelity to the structured content. We derive two datasets from the data-to-text task as our testbed, and develop a neural method with weakly supervised competing objectives and explicit content coverage constraints. Automatic and human evaluations show superiority of our approach over competitive methods including a strong rule-based baseline and prior approaches designed for style transfer.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper addresses an interesting case of text generation, interpolating between the requirements of content fidelity and style transfer. This is carried out using a hybrid *attention-copy* mechanism, which is intended to attend to style in a reference sentence and copy content over from a structured data representation. This along with data augmentation and a combined unsupervised objective comprise the learning framework. An ablation study and human evaluation are carried out, confirming that the model presented has advantages over foregoing style transfer and generation models. I think this paper should be accepted as it addresses an interesting and widely applicable scenario, and the combined unsupervised objective is a creative and well-motivated solution. It consitutes a contribution both to neural generation and style transfer methods. It is not a strong accept as I think that when introducing a new task, more in-depth evaluation and error analysis is appropriate. One challenge with this task is evaluation -- some of the scores are close enough to 50% that it would be useful to have significance of some kind calculated, more information about the annotators and inter annotator agreement, and some amount of error analysis to help give context for Table 3. Particularly, since this is a new task, the foregoing models compared would be expected to perform worse than the new model. To learn more about ways in which the new model is well adapted to the task, it could be useful to look closer into the tradeoff between fidelity and style transfer, e.g. comapring the BLEU score between y and y_hat in addition to y* and y_hat for all models. Recommendations: 1. If possible, please use clearer notation to explain the data augmentation step. It is really interesting and creative, but very hard to follow x, x*, y, y* -- please consider using subscripts like y_side or something. 2. In section 6.2, a binary transformer classifier is mentioned. Please clarify how *manipulating the content successfully* is measured, and describe the classification task in further detail.
assistant
This paper addresses an interesting case of text generation, interpolating between the requirements of content fidelity and style transfer. This is carried out using a hybrid *attention-copy* mechanism, which is intended to attend to style in a reference sentence and copy content over from a structured data representation. This along with data augmentation and a combined unsupervised objective comprise the learning framework. An ablation study and human evaluation are carried out, confirming that the model presented has advantages over foregoing style transfer and generation models. I think this paper should be accepted as it addresses an interesting and widely applicable scenario, and the combined unsupervised objective is a creative and well-motivated solution. It consitutes a contribution both to neural generation and style transfer methods. It is not a strong accept as I think that when introducing a new task, more in-depth evaluation and error analysis is appropriate. One challenge with this task is evaluation -- some of the scores are close enough to 50% that it would be useful to have significance of some kind calculated, more information about the annotators and inter annotator agreement, and some amount of error analysis to help give context for Table 3. Particularly, since this is a new task, the foregoing models compared would be expected to perform worse than the new model. To learn more about ways in which the new model is well adapted to the task, it could be useful to look closer into the tradeoff between fidelity and style transfer, e.g. comapring the BLEU score between y and y_hat in addition to y* and y_hat for all models. Recommendations: 1. If possible, please use clearer notation to explain the data augmentation step. It is really interesting and creative, but very hard to follow x, x*, y, y* -- please consider using subscripts like y_side or something. 2. In section 6.2, a binary transformer classifier is mentioned. Please clarify how *manipulating the content successfully* is measured, and describe the classification task in further detail.