We improve the informativeness of models for conditional text generation using techniques from computational pragmatics. These techniques formulate language production as a game between speakers and listeners, in which a speaker should generate output text that a listener can use to correctly identify the original input that the text describes. While such approaches are widely used in cognitive science and grounded language learning, they have received less attention for more standard language generation tasks. We consider two pragmatic modeling methods for text generation: one where pragmatics is imposed by information preservation, and another where pragmatics is imposed by explicit modeling of distractors. We find that these methods improve the performance of strong existing systems for abstractive summarization and generation from structured meaning representations. Computational approaches to pragmatics cast language generation and interpretation as game-theoretic or Bayesian inference procedures (Gol-land et al., 2010; Frank and Goodman, 2012). While such approaches are capable of modeling a variety of pragmatic phenomena, their main application in natural language processing has been to improve the informativeness of generated text in grounded language learning problems (Monroe et al., 2018). In this paper, we show that pragmatic reasoning can be similarly used to improve performance in more traditional language generation tasks like generation from structured meaning representations (Figure 1) and summarization. Our work builds on a line of learned Rational Speech Acts (RSA) models (Monroe and Potts, 2015; Andreas and Klein, 2016), in which generated strings are selected to optimize the behav-Input meaning representation ( i ): NAME [FITZBILLIES ], EATTYPE [ COFFEE SHOP ], FOOD [ENGLISH ], PRICERANGE [ CHEAP ], CUSTOMERRATING [5 OUT OF 5], AREA [RIVERSIDE ], FAMILYFRIENDLY [ YES ] Human-written A cheap coffee shop in riverside with a 5 out of 5 customer rating is Fitzbillies. ior of an embedded listener model. The canonical presentation of the RSA framework (Frank and Goodman, 2012) is grounded in reference resolution: models of speakers attempt to describe referents in the presence of distractors, and models of listeners attempt to resolve descriptors to referents. Recent work has extended these models to more complex groundings, including images (Mao et al., 2015) and trajectories (Fried et al., 2018). The techniques used in these settings are similar, and the primary intuition of the RSA framework is preserved: from the speaker's perspective, a good description is one that picks out, as discriminatively as possible, the content the speaker intends for the listener to identify. Outside of grounding, cognitive modeling (Frank et al., 2009), and targeted analysis of linguistic phenomena (Orita et al., 2015), rational speech acts models have seen limited application in the natural language processing literature. In this work we show that they can be extended to a distinct class of language generation problems that use as referents structured descriptions of lingustic content, or other natural language texts. In accordance with the maxim of quantity (Grice, 1970) or the Q-principle (Horn, 1984), pragmatic approaches naturally correct underin-formativeness problems observed in state-of-the-art language generation systems ( S 0 in Figure 1). We present experiments on two language generation tasks: generation from meaning representations (Novikova et al., 2017) and summarization. For each task, we evaluate two models of pragmatics: the reconstructor-based model of Fried et al. (2018) and the distractor-based model of Cohn-Gordon et al. (2018). Both models improve performance on both tasks, increasing ROUGE scores by 0.20.5 points on the CNN/Daily Mail abstractive summarization dataset and BLEU scores by 2 points on the End-to-End (E2E) generation dataset, obtaining new state-of-the-art results. We formulate a conditional generation task as taking an input i from a space of possible inputs I (e.g., input sentences for abstractive summarization; meaning representations for structured generation) and producing an output o as a sequence of tokens ( o 1 , . . . , o T ) . We build our pragmatic approaches on top of learned base speaker models S 0 , which produce a probability distribution S 0 ( o | i ) over output text for a given input. We focus on two conditional generation tasks where the information in the input context should largely be preserved in the output text, and apply the pragmatic procedures outlined in Sec. 3 to each task. For these S 0 models we use systems from past work that are strong, but may still be underinfor-mative relative to human reference outputs (e.g., Figure 1). Meaning Representations Our first task is generation from structured meaning representations (MRs) containing attribute-value pairs (Novikova et al., 2017). An example is shown in Figure 1, where systems must generate a description of the restaurant with the specified attributes. We apply pragmatics to encourage output strings from which the input MR can be identified. For our S 0 model, we use a publicly-released neural generation system (Puzikov and Gurevych, 2018) that achieves comparable performance to the best published results in Dusek et al. (2018). Abstractive Summarization Our second task is multi-sentence document summarization. There is a vast amount of past work on summarization (Nenkova and McKeown, 2011); recent neural models have used large datasets (e.g., Hermann et al. (2015)) to train models in both the extractive (Cheng and Lapata, 2016; Nallapati et al., 2017) and abstractive (Rush et al., 2015; See et al., 2017) settings. Among these works, we build on the recent abstractive neural summarization system of Chen and Bansal (2018). First, this system uses a sentence-level extractive model RNN-EXT to identify a sequence of salient sentences i (1) , . . . i ( P ) in each source document. Second, the system uses an abstractive model ABS to rewrite each i ( p ) into an output o ( p ) , which are then concatenated to produce the final summary. We rely on the fixed RNNEXT model to extract sentences as inputs in our pragmatic procedure, using ABS as our S 0 model and applying pragmatics to the i ( p )  o ( p ) abstractive step. To produce informative outputs, we consider pragmatic methods that extend the base speaker models, S 0 , using listener models, L , which produce a distribution L ( i | o ) over possible inputs given an output. Listener models are used to derive pragmatic speakers , S 1 ( o | i ) , which produce output that has a high probability of making a listener model L identify the correct input. There are a large space of possible choices for designing L and deriving S 1 ; we follow two lines of past work which we categorize as reconstructor-based and distractor-based . We tailor each of these pragmatic methods to both our two tasks by developing reconstructor models and methods of choosing distractors. Pragmatic approaches in this category (Dusek and Jurccek, 2016; Fried et al., 2018) rely on a reconstructor listener model LR defined independently of the speaker. This listener model produces a distribution LR ( i | o ) over all possible input contexts i  I , given an output description o . We use sequence-to-sequence or structured classification models for LR (described below), and train these models on the same data used to supervise the S 0 models. The listener model and the base speaker model together define a pragmatic speaker , with output score given by: SR 1 ( o | i ) = LR ( i | o )   S 0 ( o | i ) 1   (1) where  is a rationality parameter that controls how much the model optimizes for discriminative outputs (see Monroe et al. (2017) and Fried et al. (2018) for a discussion). We select an output text sequence o for a given input i by choosing the highest scoring output under Eq. 1 from a set of candidates obtained by beam search in S 0 (  | i ) . Meaning Representations We construct LR for the meaning representation generation task as a multi-task, multi-class classifier, defining a distribution over possible values for each attribute. Each MR attribute has its own prediction layer and attention-based aggregation layer, which conditions on a basic encoding of o shared across all attributes. See Appendix A.1 for architecture details. We then define LR ( i | o ) as the joint probability of predicting all input MR attributes in i from o . Summarization To construct LR for summarization, we train an ABS model (of the type we use for S 0 , Chen and Bansal (2018)) but in reverse, i.e., taking as input a sentence in the summary and producing a sentence in the source document. We train LR on the same heuristically-extracted and aligned source document sentences used to train S 0 (Chen and Bansal, 2018). Pragmatic approaches in this category (Frank and Goodman, 2012; Andreas and Klein, 2016; Vedan-tam et al., 2017; Cohn-Gordon et al., 2018) derive pragmatic behavior by producing outputs that distinguish the input i from an alternate distractor input (or inputs). We construct a distractor (cid:101)  for a given input i in a task-dependent way. 1 We follow the approach of Cohn-Gordon et al. (2018), outlined briefly here. The base speakers we build on produce outputs incrementally, where the probability of o t , the word output at time t , is conditioned on the input and the previously generated words: S 0 ( o t | i, o <t ) . Since the output is generated incrementally and there is no separate 1 In tasks such as contrastive captioning or referring expression generation, these distractors are given; for the conditional generation task, we will show that pragmatic behavior can be obtained by constructing or selecting a single distractor that contrasts with the input i . listener model that needs to condition on entire output decisions, the distractor-based approach is able to make pragmatic decisions at each word rather than choosing between entire output candidates (as in the reconstructor approaches). The listener LD and pragmatic speaker SD 1 are derived from the base speaker S 0 and a belief distribution p t (  ) maintained at each timestep t over the possible inputs ID : LD ( i | o <t )  S 0 ( o <t | i )  p t  1 ( i ) (2) SD 1 ( o t | i, o <t )  LD ( i | o <t )   S 0 ( o t | i, o <t ) (3) p t ( i )  S 0 ( o t | i, o <t )  LD ( i | o <t ) (4) where  is again a rationality parameter, and the initial belief distribution p 0 (  ) is uniform, i.e., p 0 ( i ) = p 0 ( (cid:101)  ) = 0 . 5 . Eqs. 2 and 4 are normalized over the true input i and distractor (cid:101)  ; Eq. 3 is normalized over the output vocabulary. We construct an output text sequence for the pragmatic speaker SD 1 incrementally using beam search to approximately maximize Eq. 3. Meaning Representations A distractor MR is automatically constructed for each input to be the most distinctive possible against the input. We construct this distractor by masking each present input attribute and replacing the value of each non-present attribute with the value that is most frequent for that attribute in the training data. For example, for the input MR in Figure 1, the distractor is NEAR [BURGERKING ]. Summarization For each extracted input sentence i ( p ) , we use the previous extracted sentence i ( p  1) from the same document as the distractor input (cid:101)  (for the first sentence we do not use a distrac-tor). This is intended to encourage outputs o ( p ) to contain distinctive information against other summaries produced within the same document. For each of our two conditional generation tasks we evaluate on a standard benchmark dataset, following past work by using automatic evaluation against human-produced reference text. We choose hyperparameters for our models (beam size, and parameters  and  ) to maximize task metrics on each dataset's development set; see Appendix A.2 for the settings used. 2 2 Our code is publicly available at https://github. We evaluate on the E2E task of generation from meaning representations containing restaurant attributes (Novikova et al., 2017). We report the task's five automatic metrics: BLEU (Papineni et al., 2002), NIST (Doddington, 2002), METEOR (Lavie and Agarwal, 2007), ROUGE-L (Lin, 2004) and CIDE r (Vedantam et al., 2015). Table 1 compares the performance of our base S 0 and pragmatic models to the baseline T-Gen system (Dusek and Jurccek, 2016) and the best previous result from the 20 primary systems evaluated in the E2E challenge (Dusek et al., 2018). The systems obtaining these results encompass a range of approaches: a template system (Puzikov and Gurevych, 2018), a neural model (Zhang et al., 2018), models trained with reinforcement learning (Gong, 2018), and systems using ensembling and reranking (Juraska et al., 2018). To ensure that the benefit of the reconstructor-based pragmatic approach, which uses two models, is not due solely to a model combination effect, we also compare to an ensemble of two base models ( S 0  2 ). This ensemble uses a weighted combination of scores of two independently-trained S 0 models, following Eq. 1 (with weights tuned on the development data). Both of our pragmatic systems improve over the strong baseline S 0 system on all five metrics, with the largest improvements (2.1 BLEU , 0.2 NIST , 0.8 METEOR , 1.5 ROUGE-L , and 0.1 CIDE r) from the SR 1 model. This SR 1 model outperforms the previous best results obtained by any system in the E2E challenge on BLEU , NIST , and CIDE r, with comparable performance on METEOR and ROUGE-L . We evaluate on the CNN/Daily Mail summarization dataset (Hermann et al., 2015; Nallapati et al., 2016), using See et al.'s (2017) non-anonymized preprocessing. As in previous work (Chen and Bansal, 2018), we evaluate using ROUGE and METEOR . Table 2 compares our pragmatic systems to the base S 0 model (with scores taken from Chen and Bansal (2018); we obtained comparable performance in our reproduction 3 ), an ensemble of two of these base models, and the best previous abstractive summarization result for each metric on this dataset (Celikyilmaz et al., 2018; Paulus et al., 2018; Chen and Bansal, 2018). We also report two extractive baselines: Lead-3 , which uses the first three sentences of the document as the summary (See et al., 2017), and Inputs , the concatenation of the extracted sentences used as inputs to our models (i.e., i (1) , . . . , i ( P ) ). The pragmatic methods obtain improvements of 0.20.5 in ROUGE scores and 0.21.8 METEOR over the base S 0 model, with the distractor-based approach SD 1 outperforming the reconstructor-based approach SR 1 . SD 1 is strong across all metrics, obtaining results competitive to the best previous abstractive systems. 3 We use retrained versions of Chen and Bansal (2018)'s sentence extractor and abstractive S 0 models in all our experiments, as well as their n-gram reranking-based inference procedure, replacing scores from the base model S 0 with scores from SR 1 or SD 1 in the respective pragmatic procedures. (a) Coverage ratios by attribute type for the base model S 0 and pragmatic models SR 1 and SD 1 . The pragmatic models typically improve coverage ratios across attribute types when compared to the base model. (b) Coverage ratios by attribute type (columns) for the base model S 0 , and for the pragmatic system SD 1 when constructing the distractor by masking the specified attribute (rows). Cell colors are the degree the coverage ratio increases (green) or decreases (red) relative to S 0 . The base speaker S 0 model is often underinfor-mative, e.g., for the E2E task failing to mention certain attributes of a MR, even though almost all the training examples incorporate all of them. To better understand the performance improvements from the pragmatic models for E2E, we compute a coverage ratio as a proxy measure of how well content in the input is preserved in the generated outputs. The coverage ratio for each attribute is the fraction of times there is an exact match between the text in the generated output and the attribute's value in the source MR (for instances where the attribute is specified). 4 Figure 2(a) shows coverage ratio by attribute category for all models. The SR 1 model increases the coverage ratio when compared to S 0 across all attributes, showing that using the reconstruction model score to select outputs does lead to an increase in mentions for each attribute. Coverage ratios increase for SD 1 as well in four out of six categories, but the increase is typically less than that produced by SR 1 . While SD 1 optimizes less explicitly for attribute mentions than SR 1 , it still provides a potential method to control generated outputs by choosing alternate distractors. Figure 2(b) shows coverage ratios for SD 1 when masking only a single attribute in the distractor. The highest coverage ratio for each attribute is usually obtained when masking that attribute in the distractor MR (entries on the main diagonal, underlined), in particular for FAMILYFRIENDLY (FF), FOOD , PRICERANGE 4 Note that this measure roughly provides a lower bound on the model's actual informativeness for each attribute, since the measure does not assign credit for paraphrases. (PR), and AREA . However, masking a single attribute sometimes results in decreasing the coverage ratio, and we also observe substantial increases from masking other attributes: e.g., masking either FAMILYFRIENDLY or CUSTOMERRATING (CR) produces an equal increase in coverage ratio for the CUSTOMERRATING attribute. This may reflect underlying correlations in the training data, as these two attributes have a small number of possible values (3 and 7, respectively). Our results show that S 0 models from previous work, while strong, still imperfectly capture the behavior that people exhibit when generating text; and an explicit pragmatic modeling procedure can improve results. Both pragmatic methods evaluated in this paper encourage prediction of outputs that can be used to identify their inputs, either by reconstructing inputs in their entirety or distinguishing true inputs from distractors, so it is perhaps unsurprising that both methods produce similar improvements in performance. Future work might allow finer-grained modeling of the tradeoff between under and over -informativity within the sequence generation pipeline (e.g., with a learned communication cost model) or explore applications of pragmatics for content selection earlier in the generation pipeline. Thanks to Reuben Cohn-Gordon for many helpful discussions and suggestions. This work was supported by DARPA through the XAI program. DF is supported by a Tencent AI Lab Fellowship.
result abstain abstain method result abstain abstain result abstain method abstain abstain abstain abstain abstain objective abstain method method abstain method method method method abstain abstain method abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain method other method abstain method abstain abstain method method method abstain method method method abstain abstain other abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain result other abstain abstain result abstain abstain abstain abstain result abstain result abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result method abstain other other other