A prevailing paradigm in neural text generation is one-shot generation, where text is produced in a single step. The one-shot setting is inadequate, however, when the constraints the user wishes to impose on the generated text are dynamic, especially when authoring longer documents. We address this limitation with an interactive text generation setting in which the user interacts with the system by issuing commands to edit existing text. To this end, we propose a novel text editing task, and introduce WikiDocEdits, a dataset of single-sentence edits extracted from Wikipedia revision histories. We show that our Interactive Editor, a transformer-based model trained on this dataset, outperforms baselines and obtains positive results in both automatic and human evaluations. We present empirical and qualitative analyses of this model's performance. 1 1 Introduction A long-standing goal of natural language processing research has been to generate long-form text (Lebowitz, 1985; Fan et al., 2018; Rashkin et al., 2020). Recent large generative language models such as GPT-2 (Radford et al., 2019), and GPT-3 (Brown et al., 2020), demonstrate an impressive ability to generate fluent text, but their outputs are difficult to control beyond a prompt, and they manifest a tendency to hallucinate facts (Wiseman et al., 2017). Much recent work has thus focused on making such models more controllable (Keskar et al., 2019; Hu et al., 2017; Zhang et al., 2020; Dathathri et al., 2019), and factually grounded (Guu et al., 2020; Liu et al., 2018b). * Work done at Microsoft Research. 1 All our code (including code to recreate our data) and pre-trained models will be made available at: http://microsoft.com/research/project/interactive-document-generation Barack Obama was the 44 th President of the United States. Most such work only considers a one-shot generation setting. Given a set of inputs, which may be a prompt, a control code (Keskar et al., 2019), or a table of data (Liu et al., 2018b) for example, the system generates text in a single step. Humans, though, often produce text through an evolutionary process involving multiple draft-edit cycles. This is not simply because they make mistakes when writing, but because they may require multiple iterations to help them shape and even make sense of what they want to express (Pirolli and Card, 2005). For example, consider a user writing an article about Barack Obama. They might start with a simple sentence such as Barack Obama was the 44th President of the United States. Next, they may wish to expand on that sentence, adding information, or rephrasing it to integrate it better with the text. Replicating this process in software will mean allowing users to adjust their requirements in response to model outputs. Even an error-free system that meets all of a user's initial requirements does not obviate the need for iteration, since those constraints are themselves dynamic. While this work focuses on text, we also note that these arguments extend to other settings where a system must generate a complex, structured object for a user, such as image or code generation. The purpose of this paper is to bring into view the task of controllable text editing, as a step beyond one-shot generation towards interactive document generation. A full interactive document generation system will likely comprise multiple components, possibly including one-shot generation to create a first draft. Editing is crucial to interactivity because it allows users to change previously generated text to fit their dynamic constraints. This is a stateful operation, where the state is the current version of the document, as opposed to stateless recasting of text from scratch using a one-shot model. While services like Gram-marly or MS Word already offer rewriting suggestions, they mainly focus on syntactic or stylistic edits such as paraphrases (Gupta et al., 2018). In this work, we are interested in a broader range of edits, particularly those that add or remove content, or change the meaning of text. Figure 1 illustrates this editing setting with an example from our trained model, where a user produces a sentence about Barack Obama over multiple edits. In sum, we make the following contributions: We introduce a challenging new text editing task, wherein a model must learn to edit text in response to a user command, while drawing on grounding to avoid problems of hallucination (Wise-man et al., 2017). To accompany this task, we release an open-source dataset of sentence-level edits extracted from Wikipedia, including editor comments, which we leverage as natural language commands, together with pre-retrieved grounding documents. We show that a transformer-based editing model trained on our data outperforms parrot and GPT-2 baselines, and obtains competitive results compared to gold-standard edits in human evaluations. We then perform an empirical analysis of our model's performance, showing the importance of the command and grounding, and the varying difficulty of edits in our dataset. We now formalize our text editing task. Let D be a document, q a user command 2 , and G some appropriate form of grounding. Moreover, let D (cid:48) be an edited version of D . Then our task is, given a dataset of edits D = { ( D 0 , q 0 , G 0 , D (cid:48) 0 ) , ..., ( DN , q N , GN , D (cid:48) N ) } , learn to produce document D (cid:48) , given D , q , and G . Note that while previous work on text editing usually only considers D as input, we include both a form of control q and grounding G . The command is needed because otherwise the type of edit to be made is undefined, while the grounding provides external knowledge needed to make an edit. In our specific instance of this task, we will only consider sentence-level edits. More formally, we consider edits D  D (cid:48) , where D and D (cid:48) differ only on a single sentence s  D , respectively s (cid:48)  D (cid:48) . While, in general, edits can vary in complexity from document-level to character-level changes, sentences are a natural way to break down text into relatively independent units of meaning, so it makes sense to edit text one sentence at a time. More complex, document-level edits can be seen as a composition of multiple sentence-level edits. Additionally, we will consider user commands q written in natural language, e.g., add years in of-fice. The command could also take other forms, such as a categorical variable, but natural language allows for the greatest flexibility in specifying what the edit should accomplish. Moreover, natural language commands are a good fit for our model, which we will initialize with pretrained language model weights. For similar reasons, we will also consider corpora of text snippets as our grounding G . Alternatively, the grounding could also consist of structured data such as tables or graphs. In a real user scenario, this grounding might be supplied by the user, or retrieved on the fly. For our dataset, we pre-retrieve groundings by querying a commercial search engine. To accompany our text editing task we present a novel dataset of nearly 12 million sentence-level edits, WikiDocEdits. These edits were extracted from the revision histories in the February 1, 2020 2 This notation reflects that the edit command is analogous to a query in a retrieval or QA setting in that it expresses a form of user intent. dump of English Wikipedia. 3 For a given Wikipedia page, a revision consists of a source and target text, corresponding to the old and new versions of the page. Each revision is also accompanied by an editor comment, which we will use as a proxy for the user command. For a given revision, we split the source and target texts into sentences and then attempt to match the sentences between source and target. For efficiency, we only look at a k -sentence neighborhood. Unmatched sentences are candidates for edits. A source sentence s and target sentence t form an edit pair s  t if f ( s, t ) > (cid:15) , where f is sentence-level BLEU 4 without smoothing and (cid:15) = 0 . 1 in our case. If an unmatched source sentence does not form an edit pair with any target sentence, we consider it to be a sentence deletion. This can also be thought of as matching to an empty sentence. We identify sentence insertions in an analogous manner. Importantly, we only consider revisions that contain a single sentence-level edit. Otherwise, the editor comment that accompanies each revision may only describe one of the possibly many sentence-level edits. See appendix A for a detailed description of our processing pipeline. We retrieve grounding snippets for the edits in our dataset by querying a commercial search engine. In order to formulate a query for a given edit, we combine the relevant page and section titles with keywords 5 from the target sentence. While the target sentence is not available at test time, we make the assumption that in a real user scenario the relevant grounding would be provided by the user. We retrieve the top 200 returned web page results and only keep the preview snippets returned by the search engine as the grounding corpus. 6 Because Wikipedia, as well as several clones, often appear in search engine results, we check for 4 -gram overlap between the target sentence and each grounding snippet, removing any snippet with more than 50% overlap. Finally, we rerank 7 the retrieved snippets using an information extraction score, and merge the ranked snippets to take the first N = 512 tokens. 3 Downloadable from https://dumps.wikimedia.org/. 4 We use BLEU-4 in all experiments of this paper. 5 See appendix B for how we identify keywords. 6 We also experimented with retrieving and parsing the HTML pages from the search but this did not lead to better end-to-end performance than just using the snippets. 7 See appendix C for details on reranking. We now provide an overview of our dataset. From 667 dump files in the February 1 st 2020 dump of Wikipedia, we extract 11,850,786 edits, and take a 1% sample of 118,818 edits to run our analyses. Table 1 presents summary statistics for our data, and in the following, we break down the edits by edit type, and present some examples. See also appendix D for an analysis of the quality of the retrieved grounding. Fluency and Content Edits We are interested in the distribution of different edit types within our dataset. In particular, we want to distinguish between fluency edits, which only affect the grammar or structure of a sentence, and content edits, which change the meaning of a sentence. We can lean on previous work to categorize edits on Wikipedia. Yang et al. (2017) create 13 edit intention categories, and train a classifier to label revisions according to the categories. We apply their classifier to our data, and group their 13 categories into fluency, content, or other edits, as reported in table 2. With the caveat that the edits were labelled automatically using a trained classifier, we see that, while fluency edits make up the majority of the edits in our data, a large proportion are content edits. Examples Table 3 presents some examples from our data. These were chosen to illustrate a variety of edits. The first example shows an elaboration edit, appending new information to the end of a sentence. The second example is a simple typo fix, while the third is changing a fact. Finally, the last example is a more complex edit to reword a sentence. We can see that there is a large variety of edits in our dataset. See table 11 in the appendix for more examples. We formalize our model, which we refer to as Interactive Editor, as a standard auto-regressive sequence to sequence model. Because our data only contains single-sentence edits, we assume that the sentence to be edited in the source document is given as an input to the model. Given a source sentence s  D , the context around s , which we will refer to as D by abuse of notation, a user command q , a grounding corpus G , and a candidate target sentence s (cid:48) , the model, f , computes f ( s, s (cid:48) , D, q, G ) = P ( s (cid:48) | s, D, q, G ) = (cid:89) i P ( s (cid:48) i | s (cid:48) <i , s, D, q, G ) , where s (cid:48) <i = { s (cid:48) 0 , ..., s (cid:48) i  1 } are the tokens preceding s (cid:48) i in s (cid:48) . We use the same encoder-decoder architecture as T5 (Raffel et al., 2020) and initialize our model with pretrained language model weights. The encoder-decoder architecture allows us to perform full attention over the inputs s, D, q , and G , while the decoder allows us to auto-regressively generate s (cid:48) . Meanwhile, initializing with pretrained weights has been shown to achieve state-of-the-art results on many NLP tasks (Raffel et al., 2020). In order to adapt T5 for our task, we represent all our inputs as sequences of tokens. We then concatenate these sequences together using separator tokens, truncating and padding them to fixed lengths. This is straightforward since all our inputs are text. See fig. 2 for reference. We also use the standard cross-entropy loss to train. We train our model on a subset of  1,020K edits from WikiDocEdits. We use a train-ing/validation/test split of 1,000K/10K/10K edits, and train for 3 epochs with a fixed learning rate of 0.0001, and a batch size of 128. We use the T5-base implementation from Huggingface (Wolf et al., 2020), and finetune all weights in the model. We validate every 200 steps and select the model with the lowest validation loss. For inference we use beam search with a beam width of 5, and keep the 5 highest ranked candidates, excluding any generation that parrots the source as this corresponds to making no edits. Metrics We consider several metrics to evaluate our model. One natural metric to consider is BLEU ((Papineni et al., 2002)). BLEU shows high correlation with human judgement on machine translation (Papineni et al., 2002; Dodding-ton, 2002). While this should not a priori transfer to evaluating different tasks, our task in fact bears a high similarity to machine translation because of how the output is constrained by the inputs. If, for example, the source sentence in an English to German translation task is Sally met Lucy, the German translation must in some way mention Sally and Lucy. Similarly, in our task, if the source sentence is Barack Obama was the 44th President of the United States, and the command is add birth date, the edit must somehow mention a birth date somewhere. Thus, in our setting, BLEU makes sense as a metric since in prin-ciple a good model output should not deviate too far from the reference. We use macro-averaged Comment added class of '13 Source Krishna attended Dartmouth College where she was a double major in government and French. sentence-level BLEU with epsilon smoothing and equally weighted n -grams, with n up to 4 . One issue with BLEU is that the source and target sentences in our task are already very similar, so a model that simply parrots back the source sentence could achieve an unduly high score. Therefore, we also evaluate model outputs by comparing the word-level edits made by the model against the reference, where a word-level edit is a tuple of an operation, either insertion or deletion, a position, and a word. For example, in the edit Barack Obama was the 44 th President of the United States  Barack Obama, born August 4 th 1961, was the 44 th President of the United States, the set of word edits would look like { ( insert , 2 ,  , ) , ( insert , 3 ,  born ) , ... } . Now, denote the set of word edits between two sentences a and b as WE ( a, b ) . Then, with s the source sentence, s (cid:48) the reference target sentence and h the target sentence generated by the model, we compute the precision PWE ( s (cid:48) , h, s ) = | WE ( s (cid:48) , s )  WE ( h, s ) | | WE ( h, s ) | , recall, RWE ( s (cid:48) , h, s ) = | WE ( s (cid:48) , s )  WE ( h, s ) | | WE ( s (cid:48) , s ) | , and F1 score, F 1 , WE ( s (cid:48) , h, s ) = 2  PWE  RWEPWE + RWE . Finally, we compute sentence-level accuracy, which reports the proportion of edits for which the model output exactly matched the reference. Baselines We use two baselines to compare our model to. First, we consider the parrot baseline that simply outputs the source sentence as is. The second baseline attempts to delete the source sentence and replace it with a new sentence. We use a pretrained GPT-2 model (Radford et al., 2019) that generates a sentence given the left context. Table 5 presents our main results. Notice that the parrot baseline is able to achieve a considerably high BLEU score, as expected, while the GPT-2 baseline surprisingly achieves a high word edit recall score. Our interactive neural editor model is able to beat both baselines across all metrics, as would be expected. Even on a harsh metric like accuracy our model achieves a nontrivial score, although we suspect most of the edits that the model gets exactly right are fluency edits. See table 6 for Comment Added more marriage info. Ablations The middle rows of Table 5 show the results for three ablations of our model. The first ablation removes everything but the source sentence s . This is similar to the paraphrase setting (Gupta et al., 2018), and the editing setting in Faruqui et al. (2018) and Yin et al. (2018). We can see that including the context, grounding, and command as additional inputs yields significant improvements over only using the source sentence. We can also see from the second ablation that the commands are a crucial element in the model's performance. This is not surprising since without a command the model must guess what type of edit to make. Similarly, the model without grounding performs considerably worse than the full model, showing that the grounding is equally important as the command. Surprisingly, the last two ablations perform only marginally better than the first, meaning that removing the grounding in addition to the commands, or vice-versa, does not lead to a large drop in performance. This seems to suggest a synergistic effect between the command and the grounding, which makes sense since the model would not know what to do with the grounding without a command, and likewise, the model would not have access to the right information without the grounding, even if it knew what to edit from the command. Breakdown by edit type The results of our full model are broken down by edit intention labels in Table 6. The columns report the same metrics as in our main table of results, with the exception of S-BLEU, which reports the BLEU score between the source sentence and target, and the last column, which reports the number of test edits that were classified into each category. With the caveat that intention labels come from an automatic classifier and not human annotation, we can observe that our model has varying performance across different types of edits. The model performs very well on fluency edits, but worse on content edits. This comes at no surprise given that fluency ed-Intention Category Acc. its should be easier as they usually correct minor mistakes, which a language model should be able to detect from pretraining. Content edits, on the other hand, require pulling the correct information from the grounding and incorporating it in the correct manner into the sentence. The S-BLEU scores confirm this since the source sentences in the fluency examples are much more similar to the target sentences than for the content edits. In fact, when looking at the absolute improvement of the BLEU over the S-BLEU scores, the model performs equally well on both types of edits. We conducted two rounds of human evaluations, each time across 200 examples from our test set. Annotators were crowd sourced, and each example was rated by seven judges for a total of 1400 judgements. 8 Command and Grounding In our first round of human evaluations we compared our model's top output from beam search to the reference edit. There were two tasks. In the first task, we asked judges to choose which system better accomplished the command q . In the second, we asked which system was more faithful to the grounding G . Table 7 presents the results. Although there is a clear preference for the Reference edits in the command-related task, 59% of judgments suggest that Interactive Editor may be equal to or better 8 The annotators were remunerated at a rate above the prevailing Seattle minimum wage at the time. than the reference. 9 In the grounding task, Interactive Editor demonstrates good correspondence with the background material. 10 Judges were further asked whether the retrieved grounding was relevant to the context D : 92.86% of judgments recorded the grounding as either Somewhat rele-vant or Very relevant. Absolute Scoring We also evaluated the overall quality of model outputs. We considered our full model, and our ablated model that only takes the source sentence as input. We also considered showing and hiding the edit commands, for a total of 4 settings. For a given setting, raters were asked whether they found each of the top 3 model outputs satisfactory. Table 8 presents the results for the top model outputs, with bootstrapped p-values for pairwise comparisons. We use a Bon-ferroni corrected  = 0 . 0125 to determine significance. Note that our full model outperforms our ablated model in the first two comparisons. Inter-9 The high percentage of Neutral judgments here may be partially attributable to other factors. Majority Neutral judgments are observed for approximately 65% of those examples that received at least one Neutral judgment. This suggests many commands may not be readily interpretable to judges. 10 Appendix E presents some additional automatic metrics to measure the faithfulness of the model to the grounding. estingly, the difference is smaller when the raters are not shown the commands. Additionally, only the ablated model is rated differently depending on whether the commands are shown. This is to be expected since the ablated model is not likely to be faithful to the commands. In addition to reporting the mean scores from the raters, we can also look at the number of examples where at least one of the top model outputs was found satisfactory by human judges (i.e. scored higher than 3). We find that, when showing the edit commands, at least one of the outputs from our full model was satisfactory in 85 . 83 % of cases versus 60 . 17 % for the ablated model. This paper focuses on the task of editing individual sentences, which we believe to be a challenging task for NLP, as it involves making nuanced changes to text according to natural language commands. We also believe this task has useful applications, particularly in speech-to-text scenarios, where it may be more convenient to speak out a command rather than edit the text directly. However, we also wish to emphasize that this task is a step towards a larger goal of interactive document generation, and that there are many interesting future directions to explore in this space. While this paper has focused on single interactions (i.e. making isolated edits to text), it would be worth modeling multiple interactions between the user and model. One can imagine that there may be a natural order in which to make edits, such as adding information at the start, and fine-tuning the language at the end. It is an open question whether or not a model could learn this. For illustration, table 9 gives an example of using our model to make several edits in order to create a sentence. Ultimately, this may look more like a dialogue than a sequence of commands coming from the user. Additionally, it would also be interesting to look at other settings where a model must generate a complex, structured object for a user, such as code, or images. We hope that our text editing task, as a first step, can demonstrate the potential for interactive generation systems, and that it will encourage the community to pursue more ideas in this space. Grounded Generation Large language models can generate fluent text (Radford et al., 2019; Brown et al., 2020; Raffel et al., 2020), but they have a tendency to hallucinate facts (Wiseman et al., 2017). Thus, several works have explored using various forms of grounding to enable models to generate factually consistent texts (Koncel-Kedziorski et al., 2019; Liu et al., 2018b; Prab-humoye et al., 2019; Liu et al., 2018a; Guu et al., 2020). Our work uses grounding to ensure that edits are factually correct, although our task differs from previous work because of the user command, which requires specific information to be retrieved from the grounding during generation. Controllable Generation While grounding can be seen as a way to implicitly control the contents of generated text, other works have explored more explicit forms of control. Hokamp and Liu (2017) and Zhang et al. (2020) use lexical constraints, while Keskar et al. (2019) and Dathathri et al. (2019) control higher level attributes of text, such as style, tone, or topic. Our task instead uses natural language commands, which can flex-ibly express different types of constraints, ranging from low-level lexical ones, to high-level topical ones. In this sense, we can also draw the parallel to dialog response generation (Ghazvininejad et al., 2018; Dinan et al., 2018), task-oriented dialog (Gao et al., 2018), or open domain question answering (Min et al., 2019; Chen et al., 2017), that also involve user responses or queries, although these tasks are not concerned with text generation in the context of document creation. Story Generation The task of Document Generation considered in our work bears similarity with work on generating long-form narratives (Jain et al., 2017). While earlier work in Story Generation focused more on plan-based architectures (Lebowitz, 1985), more recent work moved towards end-to-end approaches (Fan et al., 2018) allowing generation to be unconstrained and creative. As narratives are often aimed at particular goals expressed in terms of outlines and plans, much of the literature in Story Generation is framed as a form of controllable generation, using storylines (Peng et al., 2018), events (Martin et al., 2017; Harrison et al., 2017), plot words or word skeletons (Xu et al., 2018; Ippolito et al., 2019), plans (Yao et al., 2019), story ending (Tambwekar et al., 2019), and outlines (Rashkin et al., 2020) as various forms of constraints. Our work takes a significantly different approach, as we treat document or story generation as an iterative process that allows a human to generate a full document from scratch, but also allows constraints to be more dynamic (e.g., add nationality in Table 9 only if the system missed that the first time). Text Editing Several previous works have focused on text editing. Guu et al. (2018) generate sentences by editing prototypes taken from their training corpus, although they use editing only as a means for language modeling. Wu et al. (2019) expand upon Guu et al. (2018)'s setting, but for dialog. More related to our own setting, Faruqui et al. (2018) propose WikiAtomicEdits, a dataset of edits crawled from Wikipedia. However, they consider a much narrower definition of edits than our data does. Yin et al. (2018) use WikiAtomicEdits and propose the task of learning to represent edits, which Marrese-Taylor et al. (2020) expand using a variational approach. In contrast, we are more interested in generating edits rather than representing them. Related to Wikipedia data, Pryzant et al. (2020) also used Wikipedia revision histories to learn to debias text, whereas we considered general edits. Iso et al. (2020) propose a fact-based text editing task, but they do not consider control or other types of edits. Another related task to text editing is text paraphrasing (Gupta et al., 2018), however paraphrasing usually conserves the meaning of a sentence. While the edits we consider include meaning-preserving edits, we are mostly interested in edits that affect meaning. In this work we argued that text generation should be interactive, and, as a means towards that end, we proposed a general text editing task, where a system must edit a document in response to a user command. In our specific instance of the task we considered single-sentence edits, and we crawled a dataset of several million edits from Wikipedia that included commands, in the form of editor comments, as well as grounding documents. We then showed that training a transformer-based model on our data, while initializing with pretrained language model weights, yields encouraging results on both automatic and human evaluations. Additionally, our ablation studies showed the crucial role played by the user command and grounding. Breaking down our results by types of edits, we saw that our model not only performs well on easier fluency edits, but also on much harder content edits. Finally, we discussed future research directions for interactive document generation, as well as possible extensions to other domains such as images or code. The authors would like to thank Thomas Hofmann, as well as Sudha Rao, Matt Richardson, Zhang Li, Kosh Narayanan, and Chandra Chikkareddy for their helpful suggestions.
abstain abstain abstain objective result method abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain objective objective abstain abstain abstain abstain method method abstain method result result abstain abstain abstain method method abstain method abstain abstain abstain method abstain method method abstain abstain method abstain abstain abstain abstain method method abstain abstain abstain abstain method abstain method method abstain abstain method method method abstain abstain abstain other method abstain method abstain method method method abstain abstain abstain abstain abstain method abstain method abstain abstain abstain abstain abstain abstain abstain method method method method abstain abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain other abstain abstain abstain abstain abstain result abstain abstain result abstain result abstain abstain result result abstain abstain abstain abstain result result result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result abstain abstain method method abstain method abstain abstain method abstain abstain abstain other other objective other other method other method other other objective other other other abstain method other objective method other other method objective method result result result abstain other