A limitation of current neural dialog models is that they tend to suffer from a lack of specificity and informativeness in generated responses, primarily due to dependence on training data that covers a limited variety of scenarios and conveys limited knowledge. One way to alleviate this issue is to extract relevant knowledge from external sources at decoding time and incorporate it into the dialog response. In this paper, we propose a posthoc knowledge-injection technique where we first retrieve a diverse set of relevant knowledge snippets conditioned on both the dialog history and an initial response from an existing dialog model. We construct multiple candidate responses, individually injecting each retrieved snippet into the initial response using a gradient-based decoding method, and then select the final response with an unsupervised ranking step. Our experiments in goal-oriented and knowledge-grounded dialog settings demonstrate that human annotators judge the outputs from the proposed method to be more engaging and informative compared to responses from prior dialog systems. We further show that knowledge-augmentation promotes success in achieving conversational goals in both experimental settings. Generic responses which lack specificity have been a major issue in existing dialog models (Hosseini-Asl et al., 2020; Dinan et al., 2019a). The issue in part stems from bottlenecks in dialog models due to a limited scope of scenarios and access to limited knowledge available during training. On the other hand, encoding all possible world knowledge at training time is not feasible, and even undesirable in cases where knowledge sources are dynamically varying (Ghazvininejad et al., 2018; Majumder et al., 2020b; Zhao et al., 2020; Bruyn et al., 2020; Kim et al., 2020; Prabhumoye et al., 2021). One possible approach is to incorporate There are plenty of museums to visit around Cambridge. relevant knowledge at decoding-time. For example, in Figure 1, the user is seeking options for a fun activity around Cambridge. While the initial dialog response suggests watching a movie as an option, it does not provide any information behind that choice. We propose and evaluate an approach for unsupervised knowledge injection into a dialog model's response at decoding time 1 not addressed in any previous work. We first sample a response from the model (trained on dialog data) conditioned on the dialog context. Next, we utilize the dialog context and the sampled response to query external knowledge sources. Finally, the retrieved knowledge is used to construct a more informative and engaging response (Figure 1). A major advantage of such post-hoc knowledge injection is its flexibility in adding newer knowledge sources especially where the success of achieving conversational goals relies upon the availability of relevant knowledge. Post-hoc injection also promotes efficiency in NLP applications (Schwartz et al., 2020; Strubell et al., 2019): it mitigates the need to retrain dialog models to accommodate dynamically evolving knowledge. We experiment with two types of knowledge sources: language models, which we treat as parametric knowledge bases (Petroni et al., 2019; 1 Code: https://github.com/majumderb/poki 3140 Dialog Model  Dialog History  Post-hoc Knowledge Initial Response x d Knowledge Sources  knowledge snippets N Knowledge Fidelity for k i Dialog History  forward pass for LM f luency Entailment with  backward pass with constraints Candidate* Final Response x fi Knowledge Selection Constrained Decoding Relevance-Redundancy tradeo  to select out of snippets B NDPPN B Dialog Model  Candidate Final Responses B Rank w.r.to likelihood and linguistic diversity Final Response Ranking *for each snippet k i Figure 2: Pipeline of POKI: It first retrieves post-hoc knowledge from external sources based on dialog history and an initial response from a dialog model. Then the most relevant and diverse knowledge snippets are selected from the retrieved set. Each selected snippet is individually combined with the initial response through constrained decoding to generate a candidate final response. At last, the final response is selected via an unsupervised ranking step. Note that POKI requires no additional training. Brown et al., 2020); and user review datasets such as Yelp reviews (Hajas et al., 2014) as nonparametric knowledge sources ( 2). Since it is possible to gather a large amount of related knowledge given a query, we select a relevant and diverse (estimated via information-theoretic measures) subset of knowledge snippets using an unsupervised method (3.1). Then, a gradient-based inference approach is used to construct an updated response that incorporates the selected knowledge ( 3.2). Note that our framework does not require retraining the existing dialog modelit only relies upon updating the model's output hidden states at decoding time for unsupervised knowledge injection. We experiment with two scenarios: goal-oriented and knowledge-grounded dialog where the training data covers only a fraction of the needed knowledge. Automatic evaluation reveals that our method is capable of generating highly diverse responses in both settings. In some cases, the generated response shows high overlap with the original target response showing that our unsupervised method bridges the knowledge gap between available knowledge and human-written responses present in the existing dialog corpus. An extensive human evaluation confirms that generated responses are indeed engaging, interesting, and human-like without any loss in fluency. To pinpoint the usefulness of knowledge injection in the above settings, we design a real-time study (5.3) where users interact with our system to reach a conversational goal (e.g. planning a holiday or knowing more about the solar system). We find that external knowledge enables users to achieve their goals more efficiently. Additionally, we observe that the our approach of sub-selecting relevant but diverse knowledge leads to responses that promote success in achieving conversational goals. Our goal is to construct a dialog response by injecting knowledge (from external textual sources) at decoding time, without having to retrain the models. Consider a dialog model M from which we can sample a dialog response x d given a dialog history H . We shall refer to the response x d sampled from such a model without any decoding time knowledge injection as the initial response. However, as motivated earlier, samples from such a dialog model often lack detail. To improve such responses, we retrieve and incorporate relevant external knowledge k into the initial response. To achieve our goal, we construct a query using both dialog history H and the initial response x d , and gather a relevant knowledge candidate k from a knowledge source K . The retrieved snippet can provide useful information to the end-user to achieve the conversational goal (see 5.3). We explore both parametric (e.g querying a language model) and non-parametric (e.g. deterministic retrieval using word-overlap) ways to obtain post-hoc knowledge. Pretrained language models (PTLM) are typically trained with a vast amount of text that spans a diverse range of domains. Petroni et al. (2019); Brown et al. (2020) showed that such PTLMs can be used as a source of knowledge when queried with suitable textual prompts (e.g. Seattle is famous for ). To use PTLMs in our use-case, we construct useful prompts from dialog history and the initial response. We assemble simple prompts inspired from various knowledge-seeking situations in dialog (Shwartz et al., 2020) such as [KP] is famous for , Here is what I know about [KP] : , 3141 where [KP] is a key-phrase 2 extracted from dialog context. We use gpt2-large as the PTLM. For example, a query Here is what I know about fun things around Cambridge:" results in  There are plenty of museums to visit around Cambridge. If you love hiking, you can enjoy the trails alongside the river... " as shown in Figure 1. A complete list of prompts is provided in Appendix B. We finally rank each knowledge snippet k using the likelihood obtained from the PTLM for a concatenated input of k and dialog history and choose the most likely. External knowledge in the form of a text corpus can be used as a non-parametric knowledge source available at decoding time. Compared to parametric knowledge sources, such sources do not generate text as knowledge snippets, but offer the advantage of high quality and reliability of human written text. We consider the dialog history and the initial response as a query to retrieve relevant knowledge instances from the corpus. Next, we identify the top relevant instances in the given corpus with respect to the constructed query using cosine similarity on TF-IDF based representations (Robertson et al., 1995). Effectively utilizing the retrieved knowledge snippets to construct an enriched dialog response encompasses two major challenges. Firstly, it is not practical to use potentially hundreds of knowledge snippets obtained from the retrieval step for a single response generation. Thus, we need to find a relevant but diverse subset of the snippets. Secondly, the dialog model M is trained to condition only on the dialog context, and not on the external knowledge. Hence, to leverage the knowledge snippets, we need a decoding strategy to rewrite the initial response x d such that the resulting final response x f should closely follow the knowledge snippet to be injected without a loss in the fluency and consistency. Thus, our method requires no additional training and only assumes a language model trained on dialog context (i.e. M ). We refer to our proposed framework (Figure 2) as POKI ( Po st-hoc K nowledge I njection in Generated Dialog). At each turn, we obtain N knowledge snippets from both the parametric and non-parametric sources. We wish to select a subset of B (out of N ) relevant but diverse knowledge snippets. Thus, a high PMI score would imply a larger semantic similarity between the snippet k i and H . To account for redundancy between the snippet pair k i , k j we again use the PMI score as follows: RED ij,j>i = PMI( k i , k j ) = log  p ( k j | k i ) p ( k j )  . The redundancy score is symmetric i.e. RED ij = RED ji as PMI is a symmetric measure. We estimate probabilities (both conditional and marginal) p ( . ) in the above equations using GPT2 language model, following past work (Padmaku-mar and He, 2021). The PMI measure is often considered better than other n-gram-based overlap metrics to measure the degree of association between two sentences (Kedzie et al., 2018; Padmakumar and He, 2021). Semantically similar phrases occur in both sentences that can easily be ignored by overlap based metrics. Selection via Determinantal Point Processes. To select B knowledge snippets out of N with a relevance-redundancy trade-off, we use a subset selection process named Determinantal Point Process (DPP) (Kulesza and Taskar, 2011). DPP employs a non-uniform selection that assigns low probability to subsets (here, of knowledge snippets) that are less diverse by modeling the repulsive correlation between independently occurring datapoints (see Figure 2). We build an N  N kernel matrix D , which is real, symmetric and positive semi-definite. The diagonal entries D ii are populated by the squared relevance score of the i -th knowledge REL i and the off-diagonal entries D ij are   squared redundancy scores RED ij . We adjust  in such a way that D always remains positive semi-definite (more details in (Wilhelm et al., 2018)). To select a subset of B , a DPP assigns a probability of sampling such a subset proportional to the determinant 3142 of the submatrix DB of D , constructed using the indices of the subsetted items. The DPP probability is geometrically related to the volume of the parallelepiped spanned by the selected knowledge snippets. Diverse knowledge snippets tend to be orthogonal in their space hence span larger volume (Kulesza and Taskar, 2012). Choosing B -size submatrix from N -size D is a combinatorial problem and can become prohibitively costly when N is very high. Hence, we use a greedy method (Wilhelm et al., 2018) where we initialize the selection with the most relevant k i and subsequently select the next k j that maximizes the determinant of the resultant submatrix. Upon selecting B knowledge snippets, we want to individually inject each knowledge snippet into x d to construct a candidate final response x f at inference time. Previous works have addressed the problem of unsupervised modification of already-generated text using gradient-based decoding (Dathathri et al., 2020; Qin et al., 2020) that employs an iterative procedure consisting of a forward and a backward pass. The forward pass on the generative model (here, M ) encourages fluency of the generated text while the backward pass performs gradient ascent on certain desired constraints. Note that due to the discrete nature of x d , it is not possible to directly update it via back-propagation. Therefore, we maintain the sequence of hidden representations of each output token as z from the dialog model. Each output token x d ( t ) is realized via p ( x d ( t ) )  softmax( W z ( t ) / ) , where  is the temperature hyperparameter, W is the output embedding matrix (shared with the input), and W z ( t )  RV ( V is the size of the vocabulary). Constraints. Following Majumder et al. (2021a), we define a knowledge fidelity objective that encourages x f to be minimally different from the knowledge snippet k . We achieve this by minimizing the cross entropy loss ( CE ) between knowledge tokens k (1) , . . . , k ( T ) as labels and W z (1) , . . . , W z ( T ) as the logits. We further notice that injected knowledge can influence the generation in such a way that it contradicts with responses uttered during previous turns. Hence, we also want x f to be entailed with the dialog history H . We build an entailment classifier  ( z, H ) that predicts the probability of x f (ideally, the hidden representation z of x f ) entailing H . The classifier  ( z, H ) is a bag-of-words classification layer with hidden states z from M and fine-tuned using the DNLI dataset (Welleck et al., 2019) to predict whether the current response is entailed with previous responses or not. Decoding. In the subsequent forward and backward passes, the hidden representation z is gradually perturbed via gradient ascent on the respective objectives. During backward pass, the objective with constraints is L ( H , k ; z ) =  log  ( z, H )   CE( k, W z ) with hyperparameters  and  . We use back-propagation to update z with the gradient  z L ( H , k ; z ) while the parameters of M remain fixed. The updated latent representations of z after the backward pass are denoted as z bw . A forward pass with M is required to regularize the hidden states z toward the original dialog model objective to obtain z fw . Corresponding to the t th token, the hidden states for the t + 1 th time step are computed via a weighted addition of backward and forward hidden states, i.e., z ( t +1) =   z bw ( t ) + (1   )  z fw ( t ) where   (0 , 1) is a hyperparameter. During generation, we start by sampling the initial response x d with greedy decoding from M . The hidden states z (of x d ) are iteratively updated by alternate backward and forward passes. The final response is sampled as x f  softmax( W z/ ) . The number of iterations ( = 5 ) and the  ( = 0 . 45 ) were chosen by maximizing the Z-normalized sum of dialog model perplexity and linguistic diversity (% of distinct bigrams) in a greedy hyperparameter search. More details are in Appendix B. 3.3 Unsupervised Ranking of Candidate Final Responses Several previous works often over-generate and use an additional ranking step in order to select the final candidate in unsupervised text generation (Qin et al., 2020; Shwartz et al., 2020; Paranjape and Manning, 2021). Similarly, here we want to rank the generated candidate final responses according to the diversity of the generated text as well as the conditional likelihood of generation given the dialog history. For diversity, we measure the percentage of distinct bigrams present in the response. For conditional likelihood, we use 3143 System Acc BLEU BRTSc D-2 ENTR KCopy 70.1 4.1 62.3 3.16 2.41 SimpleTOD (2020) 70.1 15.0 79.2 0.56 0.90 SimpleTOD+ (2021) 69.8 12.1 68.1 0.81 1.11 Arranger (2021) 70.2 12.3 68.5 0.93 1.15 Rewriter (2021) 70.2 12.1 69.4 1.03 1.45 POKI 71.1 13.7 74.5 3.78 2.67 w/o Entailment 69.9 10.9 67.8 3.67 2.56 w/o Kw Fidelity 70.0 12.3 71.2 0.95 1.19 Gold 100 100 100 0.78 0.86 Table 1: Automatic metrics on the test set of MultiWoZ. the pre-trained GPT2 model to obtain the log probability when the dialog history, followed by the generated response, passed as a concatenated input. Since these two scores can have varied scale, we perform Z-normalization on the individual scores and add them to obtain a single score for ranking. The highest ranked candidate response is finally rendered to the user. We experiment with two dialog scenarios: goal-oriented and knowledge grounded. Both setups are knowledge intensive but the training data in such setups often contains only a fraction of the needed knowledge. For the goal-oriented setting, we use the Multi-domain Wizard-of-Oz (Budzianowski et al., 2018) dataset. For knowledge grounded dialog, we use the Wizard-of-Wikipedia (Dinan et al., 2019b) dataset. More details are in Appendix A. Multi-domain Wizard-of-Oz (MultiWOZ) is a multi-domain dialog dataset (we use v2.0 (Hosseini-Asl et al., 2020)) consisting of goal-oriented human-human conversations. The dataset spans seven domains (restaurant, train, attraction, hotel, taxi, hospital, police) and contains 10,438 dialogs with 13.68 average turns. Since, we do not need any training data, we only use an evaluation set (of 7K utterances). Wizard-of-Wikipedia (WoW) is a knowledge grounded dialog dataset which involves retrieving relevant knowledge from Wikipedia, reading and conditioning on it, and finally generating dialog responses (Dinan et al., 2019b). The dataset contains 201K utterances from 22K dialogues spanning 1300 diverse topics, from which we use only the test set. The associated Wikipedia knowledge base has 5.4M articles and 93M sentences. Baselines for MultiWOZ. For MultiWOZ, we consider several baselines following (Sun et al., 2021) for knowledge injection. First, we use the current state-of-the-art model, SimpleTOD, for goal-oriented dialog (Hosseini-Asl et al., 2020). Sun et al. (2021) extends SimpleTOD by adding chitchat candidates to dialog histories during training. They also have other variants that either concatenate output from SimpleTOD and candidate chitchats (Arranger) or rewrite by combining both output and chitchat snippets (Rewriter). We also have a trivial baseline (KCopy) which appends the retrieved knowledge snippet k from POKI with the initial response x d . Baselines for WoW. For WoW, we use two current-best knowledge-grounded models, KGround (Wolf et al., 2019) and BART (Lewis et al., 2020a) that concatenate the associated knowledge snippets (present in WoW) and the dialog history as inputs to generate the response with supervision. KGuide (Zhao et al., 2017) and RAG (Lewis et al., 2020b) have an additional knowledge selection step modeled by a latent variable before response generation similar to knowledge grounded models. We also use the KCopy baseline, as described for MultiWOZ. Variants of POKI. To investigate the impact of various decoding constraints in POKI, we consider the following two variants of POKIw/o Entailment and w/o Knowledge (Kw) Fidelity ( 3.2). In POKI, we use SimpleTOD as the base dialog model in goal-oriented scenarios and use BART (which is a state-of-the-art model for WoW) as the base dialog model in the knowledge-grounded scenario. For all variants of POKI, we use gradient-based inference for decoding the final response. Arguably, a system which can effectively leverage additional knowledge at decoding time should generate more diverse responses. We measure percentage of distinct bigrams as Distinct-(D-2) (Li et al., 2016) and geometric mean of entropy values of empirical frequency distributions of n-grams ( n = 1 , 2 , 3 ) as Entropy (ENTR) (Jhamtani et al., 2018) for diversity. Additionally, we report overlap between generated responses and corresponding ground truth as per BLEU and BERTScore (BRTSc). For MultiWOZ, we also report the final goal accuracy (Acc) following (Hosseini-Asl et al., 2020). MultiWOZ. Table 1 shows POKI outperforms all the baselines in terms of diversity of generated responses. More importantly, we see POKI promotes accuracy of reaching the final dialog state i.e. the goal. For ablated versions of POKI, we find the entailment constraint has little effect on diversity while dropping the knowledge adherence constraint negatively influences accuracy and diversity. All variants of SimpleTOD and all versions of POKI show departure from the results obtained by SimpleTOD on BLEU and BERTScore since all of these versions add external knowledge that were not explicitly present in the data. However, we observe that the departure is not significant and POKI achieves a much closer BERTScore to SimpleTOD compared to baselines. WoW. Despite all systems for WoW use knowledge explicitly in the knowledge-grounded dialog generation task, Table 2 shows POKI generates the most diverse responses. Similar to MultiWOZ, the knowledge adherence constraint still remains a significant factor for increasing diversity, one of the main goals of knowledge injection. For WoW, we instead see POKI outperform even BART (pre-vious SOTA) in terms of BERTScore when injected with external knowledge indicating the need of the external knowledge for modeling WoW dialogs. We conduct a comparative human evaluation with 300 samples to evaluate the quality of generated dialog responses following ACUTE-Eval (Li et al., 2019). We show a generated response from POKI to an annotator with its associated dialog history to annotate if knowledge injection makes the final response more engaging , interesting and humanlike compared to a baseline response. As sanity check, we also investigate if the response remain coherent after knowledge injection. Each sample is evaluated by two annotators 3 . MultiWOZ. Table 3 records the pairwise comparison showing POKI consistently outperforms baselines on all criteria. Responses from POKI are more engaging and interesting compared to SimpleTOD and Rewriter, demonstrating that gradient-based decoding is effective for knowledge injection. In POKI, entailment constraint mostly influences coherence whereas knowledge fidelity constraint is important for engagingness and interestingness. WoW. Table 3 shows POKI outperforms baselines that use grounding knowledge during training in all criteria showing that external knowledge can be useful even in the knowledge-grounded setting to make the conversation engaging and interesting. It also indicates the limitation of the training signal or lack of access to sufficient knowledge and 3 More details of the setup are in Appendix C. 3145  : Center of the town in Cambridge. A large gap in win percentages in favor of POKI for evaluating how humanlike' is a response when compared to state-of-the-art methods suggests knowledge injection leads to more natural conversation. Here too, both decoding constraints show similar trends to MultiWOZ. Qualitative Analysis. Figure 3 shows a conversation by POKI with a user who seeks to find restaurant options around Cambridge. We observe that in most of the turns the injected knowledge appeared as an additional justification over the initial responses making the dialog engaging and effective to reach the user's goal (also noted by human judges in 5.3). For example, in turn 3, we observe that adding the extra information about Indian cuisine helped user to reach a conclusion when their original choice of English cuisine was absent. Effect of Response Length. Qualitatively, as seen in Figure 3, responses generated by POKI are longer than those from the initial response due to the post-hoc knowledge injection. In the human evaluation sample, we found that 37% of responses from POKI are similar or smaller in length compared to responses from the best baseline. We investigate if response length acted as a confounding factor during human evaluation. Among all the cases where POKI was lost over a baseline, 45% (  2% when bootstrapped with 1000 subsets of size 50) of responses from POKI were longer than those from the comparing baseline. Among win cases for POKI, we observe 49% (  3% when bootstrapped with 1000 subsets of size 50) POKI responses were longer than those from the comparing method. This indicates that human users did not only choose longer responses as better. Relevant knowledge injection has the benefit of adding more justification to terse dialog outputs and hence influencing the task outcome positively. Mirroring observations from (Ghandeharioun et al., 2019), a real-time full conversation evaluation is needed to investigate if POKI could achieve the conversational goal any better than baselines. We recruited 60 users for this study 4 . One half of the users interacted with POKI, while the other half interacted with the best baseline model that does not augment dialog responses with external knowledge. We construct a speculative goal for each user to accomplish via the conversation. We allow users to end the conversation any time they would like and ask them whether the system helped them to reach their conversation goal along with additional comments to justify their annotation. Users who interacted with a knowledge-augmented system also asked if the system provided any knowledge that user has not explicitly asked for but indeed the extra information helped them to reach the conversational goal (Majumder et al., 2021b). Finally, we also ask if they would like to engage with the system they interacted with in future. For goal-oriented dialog, we construct speculative goals (e.g. looking for entertainment options) manually from the ground truth for 300 dialog samples. Since we are not using the underlying databases, we made sure speculative goals do not require specific information (e.g. booking availability, flight information, etc.). For knowledge-grounded dialog, we provide the intended topic of 4 More details of the participants and the study setup are in Appendix C. 3146 MultiWOZ # turns  Goal Know Would use Rewriter 8  2 69% 35% 56% POKI 4  3 86% 84% 76% WoW # turns  Goal Know Would use BART 10  2 56% 70% 48% POKI 16  3 76% 89% 71% Table 4: Real-time user study with average # of turns for successful goal completion, % of time the goal was achieved, % of success cases users were helped by an additional knowledge (Know) that was not explicitly asked to reach their goal, and if users would like to use the system in future. discussion (e.g. science fiction) present in the data; the speculative goal here is to know more about, or to have an engaging conversation about the topic. Results. First of all, we find that POKI is unanimously preferred by users compared to the baseline during the user study. More importantly, we see that when the user successfully accomplished their goal, 84% of those times they found the additional knowledge helpful in the goal-oriented setting (MultiWOZ) as compared to a baseline (Rewriter) that did not use any external knowledge. Most importantly, POKI takes significantly fewer turns for users to accomplish the goal as compared to Rewriter implicitly indicating injected knowledge (we observe high correlation, 0.67) contributes toward more efficient conversations. For the knowledge-grounded setting (WoW), both BART and POKI have access to external knowledge sources. However, 89% (compared to 70%) of success scenarios were directly influ-enced by the additional post-hoc knowledge. For knowledge-grounded dialog, a longer conversation is indicative of engagingness on a particular topic (Gopalakrishnan et al., 2019), hence users preferred to converse with POKI for more turns as compared to a BART baseline. We quote a comment from a user who found a conversation about the Korean culture with POKI was particularly engaging  Before this conversation, I had less knowledge about Korean movies and art-forms. This gave me a new perspective and a handful of popular opinions to look at it. . knowledge selection step in POKI acts an information bottleneck where the quality of the generated response directly depends on the quality of the We perform a human evaluation on 200 snippets to measure the relevance and the factual correctness in two scenarios: when we randomly select a retrieved snippet or select via DPP. In Table 5, we see that the parametric knowledge source ( gpt2-large ) generates more relevant knowledge snippets than a non-parametric one. We attribute this to 1) a large and diverse dataset (webtext) used during pretraining of gpt2 as compared to yelp reviews (restricted domains) we used for retrieval, and 2) the limited recall of relevant knowledge when using word-overlap based retrieval. However, large language models are still prone to generate non-factual knowledge. We observe that DPP-based selection in POKI is able to sub-select more factual knowledge which then positively influences the final response quality. For WoW, we also compare the selected snippets with the gold knowledge available in the dataset that in turn show high fidelity in terms of BERTScore. Time Complexity. Madotto et al. (2020) shows that iterative gradient-based decoding could be slower than generating response using single forward pass from an existing model. When we benchmark POKI in an Nvidia 2080Ti GPU, in Table 6, we see that knowledge generation (or retrieval) could be a computational bottleneck for POKI. However the greedy selection and the constrained decoding step do not add significant computational load. Furthermore, POKI's performance is comparable with PPCM (Madotto et al., 2020)a more efficient version of gradient-based decoding. The efficiency of the knowledge retrieval step can be improved with better indexing (Johnson et al., 2021) which we leave as a future work. Knowledge grounded dialog datasets such as Wizard-of-Wikipedia (Dinan et al., 2019a) and Topical chat (Gopalakrishnan et al., 2019) typically consist of dialog responses paired with relevant knowledge available as collected annotations. Hence, models trained on such datasets are restricted to the knowledge sources they were exposed to at training time. Past work (Sun et al., 2021; Majumder et al., 2020a; Su et al., 2020; Komeili et al., 2021; Adolphs et al., 2021; Ghazvininejad et al., 2018; Tuan et al., 2020; Lewis et al., 2020c; Guu et al., 2020) has looked into injecting extra knowledge sources at training time in a bid to add knowledge not available originally as paired to dialog responses. However, such approaches require re-training the model if some new knowledge source were to be used. Moreover, while previous work focuses on just improving specificity of dialog response using external knowledge, we also study the effect of additional knowledge in achieving conversational goals. Improving the diversity of dialog responses by using diversity-promoting sampling has been explored in past work (Fan et al., 2018; Holtzman et al., 2020). We use a gradient-based decoding method, building on past work in this direction (Dathathri et al., 2020; Qin et al., 2020; Madotto et al., 2020; Majumder et al., 2021a). However, we propose new objectives to inject post-hoc knowledge obtained based on already generated dialog an unsupervised knowledge injection method that has not been explored so far. We propose a framework for unsupervised knowledge injection into dialog responses. We show that knowledge can be obtained post-hoc from any knowledge sources that can improve users' ability to reach their conversational goal more effectively. In future, our idea can be generalized to setups where external knowledge can justify model's predictions such as conversational recommendation. We thank anonymous reviewers for providing valuable feedback. BPM is partly supported by a Qual-comm Innovation Fellowship, a Friends of the International Center FellowshipUC San Diego, NSF Award #1750063, and MeetElise.
abstain abstain objective method objective objective abstain abstain abstain abstain abstain abstain abstain objective objective method abstain abstain abstain other method abstain method objective result result abstain objective objective objective abstain method method abstain abstain abstain abstain abstain abstain abstain method method method abstain abstain abstain abstain method method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain method abstain abstain abstain method abstain method abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method abstain abstain abstain method abstain abstain abstain abstain method method method abstain abstain abstain method abstain method method method abstain method abstain method abstain abstain method method abstain abstain method abstain method abstain method abstain method method method abstain method method method abstain abstain method method abstain method abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other other other other objective other method objective objective objective abstain other other