This work aims to build a dialogue agent that can weave new factual content into conversations as naturally as humans. We draw insights from linguistic principles of conversational analysis and annotate human-human conversations from the Switchboard Dialog Act Corpus to examine humans strategies for acknowledgement , transition , detail selection and presentation . When current chatbots (explicitly provided with new factual content) introduce facts into a conversation, their generated responses do not acknowledge the prior turns. This is because models trained with two contexts  new factual content and conversational history  generate responses that are non-specific w.r.t. one of the contexts, typically the conversational history. We show that specificity w.r.t. conversational history is better captured by pointwise conditional mutual information ( pcmi h ) than by the established use of pointwise mutual information ( pmi ). Our proposed method, Fused-PCMI, trades off pmi for pcmi h and is preferred by humans for overall quality over the Max-PMI baseline 60% of the time. Human evaluators also judge responses with higher pcmi h better at acknowledgement 74% of the time. The results demonstrate that systems mimicking human conversational traits (in this case acknowledgement) improve overall quality and more broadly illustrate the utility of linguistic principles in improving dialogue agents. Social chatbots are improving in appeal and are being deployed widely to converse with humans (Gabriel et al., 2020). Advances in neural generation (Adiwardana et al., 2020; Roller et al., 2020) enable them to handle a wide variety of user turns and to provide fluent bot responses. People expect their interactions with these dialogue agents to be similar to real social relationships (Reeves and Nass, 1996). In particular, they expect social Figure 1: The setting for conversational rephrasing. chatbots to both use information that is already known and separately add new information to the conversation, in line with the given-new contract (Clark and Haviland, 1977). Neural generation methods for adding new information (Dinan et al., 2019; Gopalakrishnan et al., 2019; Ghazvininejad et al., 2018; Zhang et al., 2018) measure progress using metrics like engag-ingness, appropriateness and informativeness. But these metrics are too broad and provide little actionable insight to drive improvements in these systems. On the other hand, psycholinguists and sociolinguists have studied human conversations in depth and have identified fine-grained conventions, principles and contracts (Grice, 1975; Clark, 1996; Krauss and Fussell, 1996). Our first contribution is a linguistic analysis of how human conversations incorporate world knowledge. We manually annotate conversations from the Switchboard corpus to identify key traits. In particular, we find that people apply four kinds of strategies: (1) acknowledgement of each other's utterances, (2) transition to new information, (3) appropriate level of detail selection and (4) presentation of factual content in forms such as opinions or experiences. To identify deficiencies of the above types in machine-learned models, we consider a simplified task of conversational rephrasing (Figure 1), in which the factual content to be added is not left latent but is provided as a text input to the model (as in Dinan et al. (2019)), along with conversational history. Just as humans do not recite a fact verbatim in a conversation, we expect the model to rephrase the factual content by taking conversational context into account. We derive the data for this task using the Topical Chat dataset (Gopalakrishnan et al., 2019) and fine-tune a large pre-trained language model on it. Li et al. (2016); Zhang et al. (2020) use maximum pointwise mutual information (Max-PMI) to filter out bad and unspecific responses sampled from a generative language model. However, we observe that Max-PMI responses lack in acknowledgement, an essential human trait. This is because a generated response that simply copies over the new factual content while largely ignoring the conversational history can have high mutual information (MI) with the overall input. Our second contribution is a method to select responses that exhibit human-like acknowledgement. To quantify the amount of information drawn from the two contexts of new factual content and conversational history, we propose using pointwise conditional mutual information (PCMI) . We show that responses with a higher PCMI w.r.t conversational history given factual content ( pcmi h ) are judged by humans to be better at acknowledging prior turns 74% of the time. 1 Then, we use pcmi h to identify Max-PMI responses that lack acknowledgement and find alternative responses (Fused-PCMI) that trade off pmi for pcmi h . Despite a lower PMI, human anno-1 Statistically significant with p < 0 . tators prefer the Fused-PCMI alternative over the Max-PMI response 60% of the time. 1 We release annotated conversations from the Switchboard corpus (with guidelines), code for fine-tuning and calculating scores and human evaluations. 2 2 Strategiesforinformative conversations To understand strategies used by humans while talking about factual knowledge, we annotate turns in human-human conversations. We adopt and extend Herbert Clark's approach to conversational analysis. According to his given-new contract (Clark and Haviland, 1977), the speaker connects their utterances with the given information (assumed to be known to the listener) and adds new information. This builds up common ground (Stalnaker, 2002) between the two participants, defined to be the sum of their mutual, common or joint knowledge, beliefs and suppositions. We identify the following four aspects to the process of adding new information to a conversation. Acknowledgement strategies According to Clark and Brennan (1991), the listener provides positive evidence for grounding. We classify all mentions of prior context into various acknowledgement strategies. 2 https://github.com/AshwinParanjape/human-like-informative-conversations by step, connecting the given, stated information to new information. We annotate the semantic justifications for topical changes as different transition strategies. Detail selection strategies According to Isaacs and Clark (1987), speakers in a conversation inevitably know varying amounts of information about the discussion topic and must assess each other's expertise to accommodate their differences. We posit that each speaker applies detail selection strategies to select the right level of detail to be presented. Presentation strategies According to Smith and Clark (1993), presentation of responses is guided by two social goals  exchange of information and self-presentation. While we do not consider social goals in this work, we hypothesize that people talk about factual information in non-factual forms (e.g., opinions, experiences, recommendations) which we classify as various presentation strategies. Dataset We annotate part of the The Switchboard Dialog Act Corpus (Stolcke et al., 2000), an extension of the Switchboard Telephone Speech Corpus (Godfrey et al., 1992) with turn-level dialog-act tags. The corpus was created by pairing speakers across the US over telephone and introducing a topic for discussion. This dataset is uniquely useful because as a speech dataset, it is more intimate and realistic than text-based conversations between strangers. We annotate conversations on social topics which might include specific knowledge (like Books, Vacations, etc.) but leave out ones about subjective or personal experiences. Specific knowledge We define specific knowledge as knowledge that can be looked up but isn't widely known (as opposed to general knowledge that everybody is expected to know and experiential knowledge that can only be derived from embodied experiences). In this work, we are interested only in specific knowledge because it serves as a source of new information in a conversation that is hard for a language model to learn implicitly but is likely available as text that can be supplied to the system. Out of 408 annotated turns, 111 (27%) incorporate specific knowledge and account for 56% of the tokens. Acknowledgement Strategies In 70% of the turns, the speaker acknowledges the prior turn corroborating Clark and Brennan (1991). Three main strategies (Figure 2): agreement (or dis-agreement), shared experiences (or differing experience) and backchanneling account for 60% of the turns (Figure 4). In certain cases, explicit acknowledgement isn't necessary. For example, the answer to a question demonstrates grounding and serves as an implicit acknowledgement. These are categorized as N/A . Transition Strategies At the beginning of a conversation, the participants use the discussion theme to pick a topic (various transition strategies are shown in Figure 3). The decision to stay on the topic or to transition to a new one is an implicit form of negotiation and depends on the interest None Shared experience Agreement Backchannel N/AOther 29% 26% 18%15%7%5% Acknowledgement ElaborateSelf ElaborateOther Commonality Differences DiscussionThemeNone 32% 22% 21% 7% 13%5% Transition 0 50 ExperienceOpinionAnswerQuestionOthersFact.Stat. and ability of both speakers to participate. Nearly half the time, people elaborate upon the current topic (Figure 4). With a supportive listener, they might elaborate upon their own prior utterance ( self-elaboration ). Or they might signal interest in continuing the topic by elaborating the other speaker's utterance ( other-elaboration ). However, in a quarter of the turns, a participant loses interest or both participants run out of material. In that case, they transition to a new topic, implicitly justified by commonalities or differences with the current topic. If all else fails, they fall back to the discussion theme to pick a new topic. Detail-selection strategies People probe the other speaker's knowledge about an entity before diving into details. As a probing mechanism, people introduce an entity without any details ( introduce-entity ) 50% of the time. Depending on the response, details are laid out 66% of the time. Note that a turn can have both labels, i.e., it can introduce an entity for the first time or it can have details of one entity while also introducing another entity. Interestingly, in 7% of turns, an entity's name is omitted but some details are presented, creating an opening for the other speaker to chime in. Presentation strategies A single utterance can have multiple modes of presentation. A factual (objective) statement of specific knowledge is uncommon (25%) in comparison with a subjective rendering in the form of an experience (53%) or an opinion (34%) (Figure 4). The other common modes of presentation are questions (9%) and answers (16%), which often occur as adjacency pairs. We also found a few other uncommon modes (7%) such as recommendations or hypotheses based on specific knowledge. The four aspects  acknowledgement, transition, detail selection and presentation  are essential ingredients and indicative of quality conversation. They provide us with finer-grained questions amenable to human evaluation: How does the agent acknowledge? , Was it a smooth transition? , Does the utterance contain the right level of detail? , and Was the information presented as experience or an opinion? . These four aspects are also more actionable than the evaluation metrics used in prior work. They can inspire new techniques that are purposefully built to emulate these strategies. For instance, transitions can be improved with purpose-built information retrieval methods that use commonalities and differences to choose a new topic. To improve detail-selection, an agent could keep track of user knowledge and pragmatically select the right level of detail. Moreover, in their datasets, Dinan et al. (2019) and Gopalakrishnan et al. (2019) asked people to reply using knowledge snippets, but that can lead to factual statements dominating the presentation strategies. We hope that newer datasets either suggest ways to reduce this bias or not provide knowledge snippets to humans in the first place but instead post facto match utterances to knowledge snippets. In the rest of the paper, we focus on generating responses with better acknowledgements. This is because current neural generation methods perform poorly in this regard when compared with the other aspects. They fail to acknowledge prior turns and even when they do, the acknowledgements are shallow and generic (e.g., backchannel). We hypothesize that the bottleneck is not the modeling capacity, but rather our inability to extract acknowledgements. The responses are not specific w.r.t. conversational context, a prerequisite for richer acknowledgements (e.g., shared experiences). We show that selecting responses specific to conversational context improves acknowledgements and overall quality. More broadly, we are able to demonstrate the utility of our linguistic analysis in evaluating and improving a dialogue agent. Current neural generation methods typically offer short and formulaic phrases as acknowledgements: That's interesting, I like that, Yeah, I agree. Such phrases are appropriate almost everywhere and convey little positive evidence for understanding or grounding. The training corpus, on the other hand, contains richer acknowledgements, which generated responses should be able to emulate. We assume that the representational capacity of current neural models is sufficient and that out of all the sampled responses, some do indeed contain a richer form of acknowledgement. We posit that non-existent or poor sample selection strategies are to blame and that without a good sample selection strategy, improvements to the dataset, model or token-wise sampling methods are unlikely to help. We hypothesize that responses that are more specific to conversational history provide better evidence for understanding and hence contain richer acknowledgements. As a baseline sample selection strategy, we first consider maximum pointwise mutual information (Max-PMI) (as used by Zhang et al. (2020)) between the generated response and the conversational contexts (i.e., new factual content and conversational history). However, this is insufficient because it is an imprecise measure of specificity w.r.t. conversational history. Instead, we use pointwise conditional mutual information (PCMI) to maintain specificity with individual contexts and propose a combination of PMI and PCMI scores to select overall better quality responses than Max-PMI. Conversational rephrasing The choice of new factual content is a confounding factor for analysis. Hence, we define a simplified task, conversational rephrasing , where content is provided as an input. Thus, conversational rephrasing is a generation task where conversational history ( h ) and new factual content ( k ) are given as inputs and a response ( g ) is generated as the output (Figure 1). We expect the generation g to paraphrase the new factual content k in a conversational manner by utilizing the conversational history h . Base generator We fix the sequence-to-sequence model and token-wise sampling method and vary the sample selection strategy. The model is trained to take h and k as input and to generate g as the output with the language modelling loss, i.e., we minimize the token-wise negative log likelihood. During generation, tokens are sampled Response pmi ( g ; h , k ) pmi ( g ; h ) pcmi h g 1 87 18 14 g 2 150 18 4 Table 1: Measures of mutual information for generated responses from Figure 1. autoregressively from left-to-right. While sampling each token, the probability distribution is truncated using nucleus sampling (Holtzman et al., 2020) but the truncation is kept to a minimum with a high value of p for top-p sampling. Multiple diverse candidates are sampled from the base generator and now the best candidate needs to be selected. PMI for overall specificity Li et al. (2016) suggest selecting the response with maximum PMI (referred to as MMI in their work) to maintain specificity and get rid of bland or low-quality samples. Pointwise Mutual Information (PMI) between two events ( x , y ) is a measure of change in the probability of one event x , given another event y : pmi( x ; y )  log p ( x | y ) p ( x ) . We use pmi to determine the increase in likelihood of g , given h and k . A candidate generation g with higher PMI is more likely given the two contexts h and k than otherwise and is therefore considered more specific to the contexts. A low PMI value for a candidate response implies non-specificity to either context providing a clear signal for discarding it. A high PMI is necessary but not sufficient for a candidate to be specific to both the contexts simultaneously, since mutual information could come from either context. For example, g 2 (Figure 1) merely copies k but gets a high PMI score (Table 1). Whereas g 1 acknowledges prior turn and uses k but gets a lower PMI score. PCMI for contextual specificity Pointwise Conditional Mutual Information (PCMI) considers a third variable ( z ) and removes information due to z from pmi( x ; y, z ) to keep only the information uniquely attributable to y . We propose using pcmi for contextual specificity, i.e., pcmi h = pcmi( g ; h | k ) for specificity w.r.t. to conversational history h , and pcmi k = pcmi( g ; k | h ) for specificity w.r.t. new factual content k . Since acknowledgement strategies are primarily based on the history of the conversation thus far, we would expect candidates with higher pcmi h to exhibit more human-like acknowledgement strategies. As a point of comparison, consider using pmi( g ; h ) instead of pcmi h . In our setting of conversational rephrasing for informative dialogue, k topically overlaps with h . If g merely copied over the new factual content k without any reference to h , it would still have a high pmi( g ; h ) due to topical overlap but a low pcmi h . Going back to Table 1, we can see that pmi( g ; h ) is unable to distinguish between the two examples but pcmi h is. In Figure 5, the above quantities are broken down to token-level granularity. We can see that specific words that are uniquely attributable to each context are cleanly separated by both pcmi h and pcmi k . Combining PMI & PCMI for overall quality To show the utility of pcmi h in improving overall quality, we propose a heuristic method to find a more balanced response ( Fused-PCMI ) than the Max-PMI response. For every Max-PMI response with a low pcmi h , we consider an alternative that has both high pcmi h and an acceptable PMI. If such an alternative is found, we select that as the Fused-PCMI response; otherwise we default to the Max-PMI response as the Fused-PCMI response. We consider a PMI score in the top 50% of the candidate set as acceptable. To compute pcmi thresholds, we calculate quantiles based on the entire validation set and consider pcmi h in the first quartile to be low and pcmi h in the fourth quartile to be high. This approach is less susceptible to outliers, more interpretable and easier to calibrate than a weighted arithmetic or geometric mean. We derive the data for our conversational rephrasing task from the Topical Chat dataset (Gopalakrishnan et al., 2019). We use it to fine-tune a large pre-trained neural language model. This forms the base model as described in Section 3. To evaluate our proposed methods, we design three experiments and perform a comparative study with human annotators. Topical Chat Dataset This is a human-human chat dataset where crowd-workers were asked to chat with each other around certain topics. They were provided with relevant interesting facts from the Today I learned (TIL) subreddit which they could use during the conversation. TILs are are short (13 sentences), self-contained, interesting facts, most of them from Wikipedia articles. When an utterance can be matched to a TIL (based on a TF-IDF threshold of 0.12), we create an instance for the conversational rephrasing task: with the utterance as g , the two previous utterances as h and the corresponding TIL as k . We split the instances into training, validation and test sets (sizes in Section A.1) such that all utterances related an entity belong to the same set. Base Model We use the GPT2-medium model (24-layer; 345M params) pretrained on the English WebText dataset (Radford et al., 2019), as implemented in HuggingFace's TransferTransfo (Wolf et al., 2019b,a) framework. Fine-tuning is performed using the language modelling objective on the training set with default hyperparameters until lowest perplexity is reached on the validation set. During generation, we sample tokens using nucleus sampling (Holtzman et al., 2020) with p = 0 . 9 and temperature  = 0 . 9 and get candidate responses. To compute auxiliary probabilities { p ( g | h ) , p ( g | k ) , p ( g ) } for these candidates, we use separate ablation models. The ablation models are trained similar to the base model but after removing respective contexts from the training inputs. To validate our proposed methods, we do a paired comparison (on Amazon Mechanical Turk) where human annotators are shown two prior turns of conversational history and asked to choose between two candidate responses. Annotators are allowed to mark both candidates as nonsensical if the responses don't make sense. In Section A.3, we show the interfaces used to collect annotations from Amazon Mechanical Turk. Each pair of responses was compared by three annotators  we consider a candidate to be better than the other when at least two of them (majority) agree upon it. For each of the following three experiments, we compare 100 pairs of candidates generated using instances from the test set. The null hypothesis ( H 0 ) for the three experiments is that there is no difference between the methods used to generate the candidates and we hope to reject the null hypothesis in favor of the alternate hypothesis ( H 1 ) at a significance level (  ) of 0.05. Exp 1: PMI and overall quality First, we want to confirm that high PMI responses are overall better quality than randomly chosen candidates ( H 1 ). To do so, we first generate 10 responses for each instance and compare the response having maximum pmi( g ; h , k ) (Max-PMI) with a randomly chosen response from the remaining 9. We ask human annotators to pick the overall better candidate response. Exp 2: pcmi h and acknowledgement We test if responses having high pcmi h provide better acknowledgement ( H 1 ). To do so, we first sample 100 responses (larger than previous experiment) and out of all possible pairs keep those with |  pcmi h | > 15 (larger than population interquartile range; Figure 8). To control for the amount of new information being added, we pick pairs with closest values of pcmi k (recall that pcmi k denotes information uniquely attributable to k ). Such selected pairs have Median |  pcmi k | = 0 . 42 . We ask annotators to pick the response that provides better acknowledgement and select an acknowlegement span to support their claim. Exp 3: Fused-PCMI vs. Max-PMI We test if the proposed method, Fused-PCMI (that combines PMI and PCMI) selects better responses than Max-PMI ( H 1 ). For Fused-PCMI, we set low and high pcmi h thresholds to be 5 and 14 respectively based on population quartiles. For instances where the Fused-PCMI response is different from the Max-PMI response, we compare the two. We consider 10 candidate responses for each test instance and find that for around 10% of the instances the Fused-PCMI candidate is different from the Max-PMI candidate. Human annotators are then asked to pick the overall better response of the two. Based on human annotations, we are able to reject H 0 in favor of H 1 in all three experiments (Table 2) 3 : high PMI responses are overall better quality than randomly chosen candidates, responses having high pcmi h provide better acknowledgement, and Fused-PCMI selects better responses than Max-PMI. we find that PMI is useful for filtering out bad samples, but not necessarily for selecting between the good samples. When paired with a random response from the top 50% of the candidates (ranked according to their PMI), people prefer the Max-PMI response only 52% of the time (not significant). On the other hand, if the random response was in the bottom 50%, then the Max-PMI response is preferred 74% of the time. 3 In Exp 2, we ask annotators to mark text-spans that indicate acknowledgement (Table 3). If token-level pcmi h is concentrated in these spans, we have further proof that pcmi h indicates acknowledgement. Indeed, in Figure 6, we see that pcmi h is most attributable to the acknowledgement spans, followed by pmi( g ; h ) and pcmi k . Thus, pcmi h captures acknowledgements with greater specificty than pmi( g ; h ) . To understand the mechanism behind the improvement in Exp 3, we look at the distribution of samples w.r.t. pcmi k and pcmi h in Figure 7. We observe that Max-PMI responses heavily skew the distribution towards higher pcmi k , whereas Fused-PCMI responses show a more balanced improvement along both pcmi h and pcmi k . Fused-PCMI increases both pcmi h and pcmi k (medians cross 75% quartiles), indicating that the responses are simultaneously specific to both h and k . We show that samples with higher pcmi h provide better acknowledgement and Fused-PCMI improves overall quality compared to Max-PMI. Thus, by improving acknowledgements  an aspect we identified during our analysis of human strategies  we were able to improve overall quality. This demonstrates the utility of linguistic analysis for finding interpretable and actionable metrics. 2018; Gopalakrishnan et al., 2019), we expect it to generalize to any dialog setting that adds new content, e.g., experiences (Ghazvininejad et al., 2018) and personas (Zhang et al., 2018). Any dual-context language generation task where the two contexts are asymmetric in their information content can potentially benefit from PCMI. There is scope for improvement: Max-PMI still selects better responses than Fused-PCMI in 40% of the instances. This could be because it is easy for the model to copy over k and generate a high PMI response that is also fluent and accurate. Fused-PCMI encourages synthesis of acknowledgement using h and abstraction over k and it could therefore be prone to disfluencies and inaccuracies. We hope that orthogonal modeling improvements (Meng et al., 2020) reduce such effects. A cause for concern with the human evaluation is low inter-annotator agreement for Exp 1 and 3 where we ask them to pick responses with overall better quality and suitability. However, quality measurements are inherently subjective; people differ in the importance they place on different aspects such as engagement, informativeness, fluency etc., as corroborated by prior work (Finch and Choi, 2020) that shows low Cohen's kappa (0.13, 0.22) for overall quality judgements. In this work, diverse expectations from multiple annotators are captured yet subsequently averaged into overall quality. We leave it to future work to find finer-grained metrics that have high inter-annotator agreement and derive empirical weights to combine them into overall quality. system, the performance of the model also depends on other factors like user compliance and the retrieval model. In practice, we think the interplay between the four linguistic aspects is critical and needs to be explored. For instance, preliminary experiments with live conversations and an off-the-shelf retriever suggested that a bad choice of k with tenuous connections to h can make synthesis harder and lead to lower quality Fused-PCMI responses. Better retrieval models (Ren et al., 2020) that make use of transition strategies to determine k can lead to better acknowledgements. human-human informative conversations and found deficiencies in current neural dialogue systems. We proposed a PCMI-based selection strategy that selected responses with acknowledgements and higher overall quality. We hope that our work provides actionable insights and metrics for future work and more generally inspires the use of linguistic literature for grounding conversational research. We are grateful to Amelia Hardy, Nandita Bhaskhar, Omar Khattab, Kaitlyn Zhou, Abigail See, other Stanford NLP group members and the anonymous reviewers for helpful comments. This research is funded in part by Samsung Electronics Co., Ltd. and in part by DARPA CwC under ARO prime contract no. W911NF-15-1-0462. This article solely reflects the opinions and conclusions of its authors. Christopher Manning is a CIFAR Fellow.
objective abstain abstain abstain result objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective method objective method method method abstain result abstain objective objective result result abstain abstain abstain method objective abstain abstain objective abstain abstain other abstain abstain abstain objective abstain objective abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other method method method abstain abstain abstain method method method abstain method abstain abstain method abstain method abstain method method method method method method method method method method abstain abstain method method method method method abstain abstain abstain result abstain abstain abstain abstain result abstain abstain result abstain result result abstain abstain abstain abstain abstain abstain method abstain abstain abstain result abstain abstain abstain abstain abstain abstain method other other other other other other