Publicly available, large pretrained Language Models (LMs) generate text with remarkable quality, but only sequentially from left to right. As a result, they are not immediately applicable to generation tasks that break the unidirectional assumption, such as paraphrasing or text-infilling, necessitating task-specific supervision. In this paper, we present REFLECTIVEDECODING , a novel unsupervised algorithm that allows for direct application of unidirectional LMs to non-sequential tasks. Our 2-step approach requires no supervision or even parallel corpora, only two off-the-shelf pretrained LMs in opposite directions: forward and backward . First, in the contextualization step, we use LMs to generate ensembles of past and future contexts which collectively capture the input (e.g. the source sentence for paraphras-ing). Second, in the reflection step, we condition on these context ensembles, generating outputs that are compatible with them. Comprehensive empirical results 1 demonstrate that REFLECTIVEDECODING outperforms strong unsupervised baselines on both paraphrasing and abductive text infilling, significantly narrowing the gap between unsupervised and supervised methods. REFLECTIVEDECODING surpasses multiple supervised baselines on various metrics including human evaluation. Language Models (LMs) like GPT-2 (Radford et al., 2019), trained over vast unstructured data, can leverage enhanced generation methods (Holtzman et al., 2020; Martins et al., 2020; Welleck et al., 2019) to give fluent and coherent continuations to given input texte.g. news articles or stories. 1 Further results and resource are available at https://homes.cs.washington.edu/pawest/ReflectiveDecoding.html ! GPT-3 (Brown et al., 2020) takes this a step further: given a small number of examples and a well-constructed prompt, it shows remarkable performance on tasks where vast quantities of supervised data and finetuning were thought to be necessary. While this demonstrates the potential for LM-decoding in few-shot or even zero-shot out-of-the-box settings, limited access to GPT-3 and immense computational cost keep this from being a widely usable or efficient solution. Liang (2021) achieve supervised-level performance in a few-shot setting using smaller, accessible models like GPT-2. They learn a small number of task-specific vectors as a prefix to the input, without tuning the model itself. Off-the-shelf GPT-2 is capable of few-shot learning given the right setup; our work aims to push this concept further, by showing that out-of-the-box LMs can solve complex generation problems simply by using the right decoding algorithm. We introduce REFLECTIVEDECODING a novel decoding method that allows LMs to be applied to generation tasks that break the text con-tinuation paradigm, such as paraphrasing and text-infilling. REFLECTIVEDECODING requires no supervision, only two complementary off-the-shelf LMsone forward (  LM) and one backward (  LM). That means no per-task finetuning, even on unstructured text in the target domain. Inspired by the distributional hypothesis (Firth, 1957), REFLECTIVEDECODING works by generating text that might occupy the same contexts as an input. We use two LMs (  LM and  LM) to generate contexts for a given input, which implicitly capture aspects of its meaning (the contextualization step). Then, in the reflection step, we condition on this ensemble of contexts, decoding over the input with generations that are distributionally related toor replacethe input. Paraphrasing is a natural application: a good paraphrase should intuitively be compatible with the same contexts as the original text. REFLECTIVEDECODING shows strong unsupervised paraphrasing performance: On the Quora question pair dataset, we find one variant of our model (RD 30 ) outperforms unsupervised baselines on all but one metric, and supervised baselines on both the SARI metric and human evaluation. We see the same trends on the Twitter URL corpus (Lan et al., 2017). REFLECTIVEDECODING can also be applied to tasks that only replace part of the input, or generate within it, like infilling ; on  NLG (Bhagavatula et al., 2020), we outperform the best unsupervised baseline on overall quality, effectively halving the gap with supervised methods. In both applications, REFLECTIVEDECODING directly applies off-the-shelf pretrained models, without finetuning on the task or target domain. This provides evidence that off-the-shelf Language Models can excel at surprising applications, when paired with decoding algorithms designed to elicit specific kinds of information. Arrows indicate the order in which sampling functions condition on and generate tokens:  indicates generating from the left-most token to the right (left-to-right), while  indicates going right-to-left. For Language Models (LMs), this means  LM is what is often called a forward LM, while  LM is a backward LM. For our sampling function (RD), this also indicates which generated context is being conditioned on, e.g.  RD conditions on left context, extending it to the right to generate output. Currently, LM-decoding is limited to a text continuation paradigm. Given an input text s input , LM ( c | s input ) generates contexts c that might come after (forward, i.e.  LM) or before (backward, i.e.  LM) the input. LM-decoding generates outside of the input by continuing it, but many tasks require us to generate over or within the input: paraphrasing requires reformulating the input, while infilling requires inserting text in the middle of it. Reflective Decoding approaches this shortcoming by turning conventional LM-decoding around. While LM ( c | s input ) generates the kinds of contexts c the input might appear in, RD generates s that might replace s input in these same contexts. The distributional hypothesis (Firth, 1957) suggests semantically similar texts appear in similar contexts, meaning RD is also likely to sample in the semantic neighborhood of s input . Concretely, REFLECTIVEDECODING samples s that fits the same contexts as s input in 2 simple steps. We first sample many representative contexts c i that could neighbor the input, e.g. using  LM in Figure 1. This is the contextualization step . Second, in the reflection step , we generate text in the opposite direction (using  LM in Figure 1), which fits these contexts as well as s input fits them. To consider all c i 's while decoding, we ensemble the different distributions imposed by conditioning on each c i :  RD (s) = (cid:81) i  LM ( s | c i ) w i Z ( s, c, w ) (1) where Z normalizes the fraction to a proper probability distribution (see Equation 2). In essence, this Algorithm 1: Learn REFLECTIVEDECODER  RD Input: Forward language model  LM Backward language model  LM Source text s input 1: Sample contexts, c 1 ...c n c   LM ( c | s input ) 2: Initialize parameters w = w 1 ...w n c s.t. (cid:80) w i = 1 , w i  0 3: learn w = arg max w  RD ( s input ) s.t. (cid:80) w i = 1 , w i  0 Output:  RD ensemble  RD restricts generations to fit all contexts c i . Reversed  RD is the same, except it uses  LM with left contexts c i generated by  LM. By ensembling the contexts in a Product of Experts (Hinton, 2002) framework, we can generate a hypothesis s that fits the full contextual fingerprint. Yet, some contexts are more informative than others: probable but generic contexts like See the appendix for details. are not descriptive of neighboring text. We learn weights w i to prioritize contexts c i in the ensemble that are most informative for s input , by maximizing the probability of s input under Equation 1 (described in Algorithm 1). In effect, we are learning an on-the-fly autoencoder at inference time, using weighted ensembles of contexts as a representation (see 2.7, A.1). To motivate how this method functions, consider the paraphrasing example from Figure 1 with input s input = How are circulatory system tissues formed? Generated contexts reflect different aspects of s input : c 1 situates s input as a question ( This is a medical question... ), while c 2 and c 3 explore central concepts ( as with all tissue... ; about the circulatory system ). Even though each context could follow many sentences, together they form a fingerprint for s input . A sentence that could be followed by all of c 1 , c 2 , c 3 will likely be a question ( c 1 ), about tissue formation ( c 2 ), and the circulatory system ( c 3 ), and generally occupy the same semantic neighborhood as s input , e.g. How do circulatory systems form? In the case of paraphrasing, our task is to replace all of s input with something that might appear in the same contexts. Other tasks, however, might require us to replace only part of a sentence (e.g. in-context paraphrasing) or even insert text at a given position (e.g. infilling). REFLECTIVEDECODING makes this easy: simply hold part of s input static when we generate from RD. 2.3 REFLECTIVEDECODING Here we dive into the details of REFLECTIVEDECODING , by considering the right-hand context ensemble (  RD), keeping in mind that the process is repeated on the left-hand as well (  RD). First, in the contextualization step (line 1 of Algorithm 1), we sample many right-hand contexts c i for s input , using  LM. These will be used as a representative sample of the contexts s input appears in. Second, in the reflection step (lines 2 & 3) our goal is to construct a sampling function  RD that will yield texts similar to s input . We define  RD as:  RD ( s ) = (cid:81) i  LM ( s | c i ) w i (cid:81) | s | j =0 (cid:80) t  V (cid:81) i  LM ( t | s j +1: | s | + c i ) w i (2) This is equivalent to Equation 1, but giving the exact normalization factor in the denominator. Equation 2 is a token-wise Product of Experts model, that captures the semantic neighborhood of s input via the combination of contexts c i and their weights w i (2.7). We learn w i that maximize  RD ( s input ) (probability of generating s input under  RD), thereby up-weighting contexts specific to s input . We initialize these weights (line 2), then train them (line 3) using the Adam optimizer (Kingma and Ba, 2014). We normalize weights into a proper probability distribution at every step. Reverse-direction  RD is learned symmetrically, flipping the roles of  LM and  LM and sampling left-hand context instead (see B.1 for details). Finally, we generate from  RD (and  RD), sampling outputs that would appear in the same contexts as s input . Depending on the application, we rank and select a final output in different ways, always using  LM and  LM together to capture bidirectional fit. Weight Learning and Pruning Context weights w i are learned using the Adam optimizer (Kingma and Ba, 2014). In practice this takes under 100 steps (negligible time compared to LM decoding). While we sample tens of contexts (line 1 of Algorithm 1), many end up with negligible weight under the learned distribution (Equation 2). To efficiently sample from  RD and  RD, we drop all but the top k c contexts and renormalize weights: k c < n c contexts are used during the reflection step. Parameters We sample n c contexts to describe the source s input . We use nucleus sampling (Holtz-Task: ! man et al., 2020) with parameter p c , and a maximum length of len c . Once  RD and  RD are learned, we sample n s generations from each, of length len s . We again use nucleus sampling, but choose p s per-example to account for vastly different entropy in RD (B.3). Values for all hyperparameters are available in B.4. Language Models We train large forward (  LM) and backward (  LM) Language Models based on GPT-2 (Radford et al., 2019) using the OpenWeb-Text training corpus (Gokaslan and Cohen, 2019) 2 . Our implementation details follow those of past work retraining GPT-2 (Zellers et al., 2019). To paraphrase, we begin by generating candidate outputs. Following 2.3 the REFLECTIVEDECODING sampling function is learned in each direction (  RD,  RD) using the source sentence s input . Then, n s generations are sampled from both  RD and  RD: s 1 , ..., s n s   RD , s n s +1 , ..., s 2  n s   RD This gives a robust set of candidates that are compatible with the same left and right contexts as s input . Many of these will be semantically related to s input , but must be scored and ranked in order to select true paraphrases. REFLECTIVEDECODING is based on the notion that good fit with the same contexts is a robust measurement of similarity, yielding a natural contextual scoring function (Equation 7 and 2.7). We measure how likely candidate s is to generate the same contexts that s input did when constructing  RD and  RD: score ( s ) = 1 n c (cid:88) c rh  LM ( c rh | s )+ 1 n c (cid:88) c lh  LM ( c lh | s ) (3) where c rh are the generated contexts used in  RD, 2 https://github.com/yet-another-account/openwebtext and c lh for  RD. This explicitly estimates how similar the contexts of s and s input are on both sides, the underlying objective of REFLECTIVEDECODING . Abductive natural language generation (  NLG from Bhagavatula et al. 2020) is the task of fill-ing in the blank between 2 observations o 1 and o 2 , with a hypothesis h that abductively explains them. The challenge for LM-decoding is making use of context from both sides ( o 1 on the left and o 2 on the right). This is particularly challenging for unsupervised decoding methods because unidirectional LMs cannot naturally condition on both sides when generating h . REFLECTIVEDECODING simplifies this problem by capturing information about both o 1 and o 2 in a single decoding function (  RD or  RD), then holding o 1 and o 2 static at generation time (i.e. teacher forcing). Concretely, we use concatenated o 1 + o 2 as s input in Algorithm 1, and construct sampling functions  RD,  RD informed by both observations. We are interested in sampling in between o 1 and o 2 , so when sampling hypotheses h from  RD we condition on the right-side observation o 2 (and vice-versa for  RD and o 1 ). This is equivalent to appending the given observation to sampled contexts: h 1 , ..., h n  s   RD ( h | o 2 ) h n  s +1 , ..., h 2  n  s   RD ( h | o 1 ) (4) Note that both  RD and  RD contain information about both o 1 and o 2 , effectively turning a 2-sided contextual constraint into a 1-sided one. We also use a task-specific scoring function to rank sampled hypotheses. We would like a hypothesis h that best explains both observations, and so use Language Models to measure this: score ( h ) =  LM ( o 1 | h + o 2 )+  LM ( o 2 | o 1 + h ) (5) Adding h should help to explain each observation given the other, i.e. that o 2 follows from o 1 + h and o 1 from h + o 2 . To filter hypotheses that only explain one of the two observations, we remove any that make either observation less probable than the empty hypothesis, imposing:  LM ( o 1 | h + o 2 ) >  LM ( o 1 | o 2 )  LM ( o 2 | o 1 + h ) >  LM ( o 2 | o 1 ) 2.7 Intuitions and Theory Here we discuss the theoretical intuition for REFLECTIVEDECODING , as a way to sample generations that share contextual fit with a source text, deriving the sampling function of Equation 2. We start by considering how to relate the meaning of two texts, generation s and input s input . We follow a distributional intuition (Firth, 1957), that meaning can be understood through the contexts in which text appears. Many distributional approaches learn contentful neural representations by predicting context given input text (Mikolov et al., 2013; Kiros et al., 2015), then compare these representations to establish semantic similarity. We can, instead, compare contexts directlyjudging the difference in meaning between texts s input and s by their divergence: DKL (  LM ( c | s input ) ,  LM ( c |  s )) (6) We use  LM to interchangeably denote the theoretical left-to-right distribution of text, and the LM estimating it. Thus,  LM ( c | s ) is the distribution over right contexts c given sentence s, and Equation 6 can be understood as the contextual information difference we expect s to have from s input . Note, we could similarly use left-hand context and  LM and do so in practice. We use finite-sample cross entropy as an effective empirical proxy for DKL :  H (  LM ( c | s input ) ,  LM ( c | s )) = 1 N (cid:88) c i  LM ( c | s input )  log  LM ( c i | s ) (7) Where c i   LM ( c | s input ) indicates sampling contexts for s input from  LM. Intuitively, we want to minimize this score when generating s : an optimal output has a similar meaning to s input and so fills approximately the same contextual hole, minimizing the value of this contextual distance. In this form,  H compares 2 complete texts s and s input but we are trying to generate s for which the divergence from s input is low. We flip the role of text and context 3 to define a function from which we can sample s :  RD ( s j | , s j +1: n ) = (cid:81) i  LM ( s j | s j +1: n + c i ) w i (cid:80) t  V (cid:81) i  LM ( t | s j +1: n + c i ) w i (8) (equivalent to Equation 2, derived in A.1) s j is the j th token in s (sampled right-to-left from n to 0), and V is the vocabulary. Weights w i are learned by maximizing the probability of s input . Equation 8, estimates the probability of predicting s input and s from a finite set of contexts c i generated from s input . This approximately minimizes Equation 6, as being generated by the same weighted ensemble of contexts strongly correlates with generating the same contexts in the same proportions, i.e. low divergence, due to the sparsity of language. We can sample s with low contextual distance from s input using  RD. Further, we can use left context to construct  RD by simply reversing the directions of the LMs used. Task: Following past work, we test our paraphrasing method (2.5) on the Quora question pair dataset. We hold out 1000 examples for testing, with the rest for training and validation (used by supervised baselines), disallowing overlap with the test set. We test a subset of models (compatible unsupervised models, MT) on the Twitter URL corpus (Lan et al., 2017), using 1000 examples from the canonical test split. Metrics: Following past work, we include automatic metrics BLEU (Papineni et al., 2002), METEOR (Denkowski and Lavie, 2014), and TER p (Snover et al., 2009). These measure agreement with references, but high reference/input overlap means copying is rewarded (Mao and Lee, 2019); indeed, copying source sentences as-is wins on these metrics (Table 1), meaning both BLEU and METEOR can be easily gamed. 3 Context is a symmetric relation: a given text serves as the one-sided context of its own context. Past work has emphasized the important challenge of generating novel paraphrases (Liu et al., 2010; Chen and Dolan, 2011) We address this in 3 ways. First, we explicitly quantify a simple notion of novelty: Novelty ( s ) = 100  BLEU ( s, s input ) (9) to quantify the novelty-quality trade-off. Second, we include the SARI metric (Xu et al., 2016) which explicitly balances novelty from input with reference overlap. Third, we quantify an overall human quality metric accounting for this. We have humans evaluate fluency, consistency, and novelty on Amazon Mechanical Turk. The overall score (Human in Table 1) is the rate examples meet thresholds for all 3: fluent enough to understand, with at most minor differences in meaning and at least minor differences in wording. On quora, we test 200 examples, with agreement (Fleiss'  Fleiss, 1971) of 0.40 (fluency), 0.54 (consistency), 0.77 (novelty) and 0.48 (over-all) i.e. moderate to substantial agreement (Landis and Koch, 1977). On the Twitter corpus, we use 100 examples with agreement of 0.39, 0.42, 0.54, and 0.36, indicating fair to moderate agreement. On both we have 3 raters per example. See C.2 for more. Baselines: Parameters for REFLECTIVEDECODING are given in B.4. We mainly compare against 3 unsupervised baselines: Controlled Sentence Generation by Metropolis Hastings (CGMH from Miao et al. 2019), Simulated Annealing (UPSA from Liu et al. 2019) and the residual VQ-VAE of Roy and Grangier (2019a) (R-VQVAE). This is a cross-section of recent approaches (VAE, editing). We also compare against a machine-translation approach (see Sec 6), pivoting through German using Transformer (Vaswani et al., 2017) models trained on WMT19 data (Barrault et al., 2019). MT is included in a separate section in our results as it uses supervised bilingual data (Table 1). We include supervised baselines: the pointer generator trained by imitation learning (PG-IL) as in Du and Ji (2019), the diversity-promoting DiPS model (Kumar et al., 2019), and a finetuned BART model (Lewis et al., 2019), which uses a more complex pretraining method than our LMs. Note that DiPS generates multiple diverse paraphrases so we pick one at random. CGMH and REFLECTIVEDECODING both return multiple sampled, ranked paraphrases. We can easily control for Novelty by taking the highest-ranked output that meets a Novelty threshold. For both, we have a version with no threshold ( T op ), and with thresholds such that average Novelty is 30 and 45 . Novelty cutoffs do not depend on the reference, only the source, and are equivalent to selecting with BLEU-ori ( Novelty is 100  BLEU-ori) by Miao et al. (2019) or Bao et al. (2019). Task: The Abductive natural language generation task (  NLG) presented in Bhagavatula et al. (2020) requires generating a hypothesis that fits We apply REFLECTIVEDECODING to this problem as outlined in 2.6, using the given data splits. Metrics: For human evaluation, over 200 examples we ask 3 raters on Amazon Mechanical Turk about coherence between h and o 1 , o 2 , o 1 + o 2 , and overall quality on 4-value likert scales. We found Fleiss' kappa (Fleiss, 1971) of 0.32, 0.40, 0.41, and 0.41 respectively, indicating fair to moderate agreement (Landis and Koch, 1977). Baselines: Parameters for REFLECTIVEDECODING are given in B.4. We include baselines from the original work: different supervised variants of GPT-2 large with access to the observations, and optionally commonsense embeddings or generations from COMET (Bosselut et al., 2019). We include unsupervised baselines of GPT-2 conditioned on o 1 + o 2 directly, the gradient-based DeLorean model of Qin et al. (2020), and ILM infilling model of Donahue et al. (2020), representing recent unsupervised methods. Paraphrasing First, the Quora dataset: On automatic metrics from past works (BLEU, METEOR, TERP ) our lowestNovelty model setting (RD Top ) achieves the highest unsupervised scores, and highest overall on BLEU. Other high scoring rows (Source, PG-IL) are similarly lowNovelty . The SARI metric explicitly balances Novelty with similarity to reference. On SARI we see such low-Novelty models perform worse. The best overall model on SARI is our mediumNovelty setting (RD 30 ) which outperforms MT and supervised models. Our human evaluation measures what fraction of outputs are found to be fluent, consistent, and novel. As with SARI, both our mid and highNovelty models perform quite well, again with the medium-Novelty setting outperforming all baselines. As further validation for SARI as a proxy for human, they share the same top-5 models. Results on the Twitter URL corpus largely support those on Quora. REFLECTIVEDECODING achieves the best unsupervised scores on novelty-aware metrics (Table 2), with the best overall SARI, even outperforming reference on the human metric, although MT achieves the highest overall. In sum, REFLECTIVEDECODING is able to compete on previously used quality metrics favoring lowNovelty , but can produce more varied outputs preferred by humans. RD 45 is among the best models by SARI and Human on Quora despite exceeding the novelty of even the reference. NLG Results on  NLG (Table 3) present a strong case that REFLECTIVEDECODING can effectively use bidirectional context. Strong hypotheses use information from both initial the observation o 1 and the future observation o 2 . Humans ranked the ability of REFLECTIVEDECODING to capture this 42.4, about 17 points above the next-best unsupervised baseline and only 15 points below the best supervised method tested. We see similar results for overall evaluation. A likely factor in this is the (comparatively) high degree of coherence between h and o 2 by REFLECTIVEDECODING . Where other methods seem to pay more attention to observation o 1 (the o 2 column generally has much lower values), REFLECTIVEDECODING has comparably high coherence with left-hand ( o 1 ) and right-hand ( o 2 ) contexts. We also include example generations in Figure 2 to demonstrate the ability of REFLECTIVEDECODING to combine o 1 and o 2 . For example, h = He put her on the swing, and while she was on the swing, she fell off and was lying on the ground. incorporates information from both observations. Specifically, it takes into account the swing that Ray is building for his daughter which is only mentioned in o 1 , and hypothesizes about a potential injury due to Ray checking on his daughter in o 2 . See appendix for more generations. Overall, the strong performance of REFLECTIVEDECODING on  NLG shows that unsupervised generation with context ensemble applies to infilling in addition to paraphrasing. REFLECTIVEDECODING Out-of-the-Box A major advantage to applying REFLECTIVEDECODING is ease-of-use: armed with our pretrained language models, practitioners can immediately begin generating. With general pretrained models and underlying principles that are domain-agnostic, REFLECTIVEDECODING works across a broad range of text styleno finetuning requiredmaking exploration and adaptation simple. Multiple rounds of generation mean REFLECTIVEDECODING may run slower than other methods at inference time 4 , but it avoids training time. There are clearly settings that favor supervised learning (narrow, known domain with abundant training data), but REFLECTIVEDECODING is a good option to begin generating and exploring immediately with high quality generation. A useful abstraction for understanding REFLECTIVEDECODING for current applications is prompting, i.e., writing a prefix to implicitly or explicitly describe a task for a pretrained model. REFLECTIVEDECODING generates natural contexts that the desired generation would appear in. This breaks from other methods of automatic prompting, which often forego natural prompts (Shin et al., 2020; Reynolds and McDonell, 2021), even making them continuous (Li and Liang, 2021; Ham-bardzumyan et al., 2021; Lester et al., 2021; Qin and Eisner, 2021). REFLECTIVEDECODING also notably creates a set of prompts (contexts) for each example, where other methods attempt to learn an overall task prompt. Still, all of these are connected by the popular intuition that useful behavior in pretrained models can be induced through contextual input. Future Applications REFLECTIVEDECODING can extend beyond our experiments here, however. A simple example is in-context paraphrasing, i.e. writing a paraphrase that fits the true context that the original sentence appears in. Most existing paraphrasing methods consider only out-of-context sentences, and would require significant changes to consider context as a constraint; for REFLECTIVEDECODING we can simply combine true and generated contexts without with the same algorithm. poten-4 Depending on parameters we found most baselines took multiple seconds per example vs. 10s of seconds for REFLECTIVEDECODING on a multi-gpu machine. tial for future work. Pretrained LMs capture rich information about text spans, but accessing it without fine-tuning is nontrivial; within the model it is an uninterpretable mass of parameters and activation weights. Our work observes that unidirectional LMs are only capturing this information to predict adjacent contextthis is the sole learning signalso all of this information is expressed in the model's context prediction. Thus, we capture some of this rich information to represent spans, by capturing a finite-sample version of this full predictive distribution in generated contexts. In REFLECTIVEDECODING specifically, we use this form of representation to generate back into the source spanparaphrasing or infillingbut the notion can be applied much more generally. In translation for instance, we might first generate contexts for the source sentence that represent its meaning, noisily translate these contexts, then impose that any translations for the source fit the same contexts under a translation-language LM. Constraining translations in this way can add robustness to existing systems by anchoring translations to informative contexts. Beyond explicit generation even, we might use a very large LM like GPT-3 to define a strong scoring function or metric as in Equation 7, first generating contexts for some target sentence, then scoring candidates by how well they generate these same contexts. As in our work, such a score could indicate how well the option fills the same contextual role as the target, harnessing the strong reasoning of whatever model is used. Distributional Intuitions A key aspect of REFLECTIVEDECODING is using a distributional intuition to represent the meaning of a text through many contexts. Kiros et al. (2015); Miao et al. (2019) quantify semantic relationships and Lin and Pantel (2001) identify paraphrastic relationships under similar intuitions. A major point of difference between past work and ours is that we sample explicit contexts, allowing unsupervised generation back from these contexts, while past work typically learns a neural representation based on contexts and conditions on this vector-encoded representation. Unsupervised Paraphrasing Some approaches train neural variational auto-encoders unsupervised to represent source sentences, then decodes from these representations to paraphrase (Roy o 1 o 2 o 1 + o 2 all Human 86.3 89.1 85.1 84.4 Supervised COMeT Emb +GPT2 69.3 60.1 56.4 56.3 COMeT Txt +GPT2 68.9 54.8 51.9 50.6 O 1 -O 2 -Only 69.2 57.7 54.3 53.8 Unsupervised GPT2-Fixed 20.6 13.9 10.8 10.3 DeLorean 48.7 24.6 23.6 22.5 ILM 45.9 27.3 25.3 25.0 Reflective Decoding 53.4 51.7 42.4 41.9 Table 3: Model performance on  NLG. and Grangier, 2019b; Bao et al., 2019). This requires training specialized representations, whereas REFLECTIVEDECODING applies general-purpose LMs. We compare to Roy and Grangier (2019b). Paraphrasing by editing the input (Miao et al., 2019; Liu et al., 2019) has shown promise. Like REFLECTIVEDECODING , these approaches can be applied without training specialized models, but are necessarily limited by edit-paths and local minima, as edits are often restricted to single-word replacement, insertion, and deletion. Generated paraphrases must follow a continuous local edit path, while REFLECTIVEDECODING can generate new sentences from scratch. REFLECTIVEDECODING and MT-based paraphrasing both pivot through an alternative textual form to paraphrase (context and translation, re-spectively). But MT paraphrasing systems cycle-translate through a pivot language (Federmann et al., 2019; Wieting and Gimpel, 2018), which requires supervised bilingual translation data, with an implicit notion of interlingual paraphrasing. Novelty in Paraphrasing Mao and Lee (2019) observe that paraphrases close to the source often win on automatic quality metrics. However, dissimilarity from the source correlates with human notions of paraphrasing (Liu et al., 2010). Kumar et al. (2019) increase novelty through their diversity-promoting sampling method. Alternative metrics that consider novelty alongside quality have been proposed (Sun and Zhou, 2012; Feder-mann et al., 2019). The SARI metric (Xu et al., 2016), included here, combines these notions. Abductive Text Infilling  NLG (Bhagavatula et al., 2020) is a text infilling task that specifically measures the ability of models to explain bidirectional context (observations o 1 , o 2 ) with a hypothesis that fits between them. This naturally fits REFLECTIVEDECODING , which fills in contextual gaps. Recent work has directly addressed this task (Qin et al., 2020) while the infilling literature is also quite applicable (Donahue et al., 2020). We compare to both of these methods on abductive infilling, showing superior results. We present REFLECTIVEDECODING , a novel unsupervised text generation method for tasks that do not fit the text continuation paradigm. It uses just two pretrained Language Models to generate contexts that capture aspects of input text, generating back into the input from there. It significantly outperforms unsupervised baselines in quality and novelty for paraphrasing. Further, in abductive natural language generation it outperforms unsupervised baselines by a significant margin and halves the gap with supervised models. REFLECTIVEDECODING uses the concept of representing meaning with generated contexts, offering new possibilities for unsupervised conditional text generation. We thank anonymous reviewers for many helpful comments. This research is supported in part by the Natural Sciences and Engineering Research Council of Canada (NSERC) (funding reference number 401233309), DARPA CwC through ARO (W911NF15-1-0543), DARPA MCS program through NIWC Pacific (N66001-19-2-4031), the Allen Institute for AI, and a gift from Intel Labs Cognitive Computing Research. In order to complete our human evaluation we used Amazon Mechanical Turk. We estimated the range of times we expected our task to take, and made sure that at minimum workers would be paid a wage of $15.00 per hour if they were solely completing our task. As part of this effort, we plan to release our code and model. Our forward and backward language models are the same size as the publicly available GPT-2 (Radford et al., 2019). Training time/energy was likely significantly smaller than the original release; existing code and hyperparameters were available, and we use a smaller dataset. Further, there is no publicly available backward GPT-2 model that we are aware of, so releasing a pair of forward and backward models that were trained on the same data allows for proper comparisons about left-to-right vs. right-to-left processing of English text. We estimate that the potential dangers of releasing this from a malicious generation perspective are low. Our forward model is similar to already released GPT-2 models. While the backward model adds new generation potential and scientific novelty, it is unlikely to compare to GPT-3 (Brown et al., 2020) which many hobbyists and private companies now have access to. We believe that releasing a pair of forward and backward models will be more useful to researchers who wish to study the symmetries and asymmetries of the linguistic distribution.
abstain abstain objective abstain objective method abstain abstain abstain other abstain abstain abstain abstain objective objective abstain abstain abstain method abstain abstain result result result abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain method abstain method method other abstain method abstain method method abstain abstain abstain abstain method method method method abstain method method abstain method method method abstain method method abstain abstain method method method method abstain method method abstain method method method abstain abstain abstain abstain other abstain abstain abstain abstain abstain method method other method method method method method abstain method method method method method method method abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain result abstain abstain result result abstain result abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result abstain abstain result method method abstain abstain abstain method other other method other other other method other other other other other other other other other other other other other abstain objective abstain abstain abstain abstain other other method method abstain abstain method method abstain abstain abstain method