We focus on the study of conversational recommendation in the context of multi-type dialogs, where the bots can proactively and naturally lead a conversation from a nonrecommendation dialog (e.g., QA) to a recommendation dialog, taking into account user's interests and feedback. To facilitate the study of this task, we create a human-to-human Chinese dialog dataset DuRecDial (about 10k dialogs, 156k utterances), which contains multiple sequential dialogs for every pair of a recommendation seeker (user) and a recommender (bot). In each dialog, the recommender proactively leads a multi-type dialog to approach recommendation targets and then makes multiple recommendations with rich interaction behavior. This dataset allows us to systematically investigate different parts of the overall problem, e.g., how to naturally lead a dialog, how to interact with users for recommendation. Finally we establish baseline results on DuRecDial for future studies. 1 1 Introduction In recent years, there has been a significant in-crease in the work of conversational recommendation due to the rise of voice-based bots (Chris-takopoulou et al., 2016; Li et al., 2018; Reschke et al., 2013; Warnestal, 2005). They focus on how to provide high-quality recommendations through dialog-based interactions with users. These work fall into two categories: (1) task-oriented dialog-modeling approaches (Christakopoulou et al., 2016; Sun and Zhang, 2018; Warnestal, 2005); (2) non-task dialog-modeling approaches with more freeform interactions (Kang et al., 2019; Li et al., 2018). Corresponding author: Wanxiang Che. 1 Dataset and codes are publicly available at https://github.com/PaddlePaddle/models/tree/develop/PaddleNLP/Research/ACL2020-DuRecDial. Almost all these work focus on a single type of dialogs, either task oriented dialogs for recommendation, or recommendation oriented open-domain conversation. Moreover, they assume that both sides in the dialog (especially the user) are aware of the conversational goal from the beginning. In many real-world applications, there are multiple dialog types in human-bot conversations (called multi-type dialogs ), such as chit-chat, task oriented dialogs, recommendation dialogs, and even question answering (Ram et al., 2018; Wang et al., 2014; Zhou et al., 2018b). Therefore it is crucial to study how to proactively and naturally make conversational recommendation by the bots in the context of multi-type human-bot communication. For example, the bots could proactively make recommendations after question answering or a task dialog to improve user experience, or it could lead a dialog from chitchat to approach a given product as commercial advertisement. However, to our knowledge, there is less previous work on this problem. To address this challenge, we present a novel task, conversational recommendation over multi-type dialogs, where we want the bot to proactively and naturally lead a conversation from a non-recommendation dialog to a recommendation dialog. For example, in Figure 1, given a starting dialog such as question answering, the bot can take into account user's interests to determine a recommendation target (the movie < The message > ) as a long-term goal, and then drives the conversation in a natural way by following short-term goals, and completes each goal in the end. Here each goal specifies a dialog type and a dialog topic. Our task setting is different from previous work (Chris-takopoulou et al., 2016; Li et al., 2018). First, the overall dialog in our task contains multiple dialog types, instead of a single dialog type as done in previous work. Second, we emphasize the initiative of the recommender, i.e. the bot proactively plans Name:  (Fanyu Yang) Gender:  (Male) Age: 20 Domains that the user likes: movie, music Stars that the user likes:  (Xun Zhou)  (Rene Liu) Recommendation accepted: <  >(Stolen Life) Recommendation rejected: <  >(The little prince) User profile Conversational recommendation Knowledge graph  <  > (Stolen life)  (movie star)  <  >(The message)  (Xun Zhou)   (the best actress of the Asian Film Awards)  (the most popular actress of the Golden Eagle Award of China TV)  ,   (It has refined characters and capricious plots.)  (Rene Liu)  <  1937> (Don't cry, Nanking!) actor c o mm e n t actor  (It shows the director's thinking on war, human nature.) c o mm e n t comment type type  (Historical war film) Goal planning: QA about <Stolen life>, chitchat about Xun Zhou, recommending the movie <The message>, recommending <Don't cry, a goal sequence to lead the dialog, and the goals are unknown to the users. When we address this task, we will encounter two difficulties: (1) how to proactively and naturally lead a conversation to approach the recommendation target, (2) how to iterate upon initial recommendation with the user. To facilitate the study of this task, we create a human-to-human rec ommendation oriented multi-type Chinese dial og dataset at Bai du ( DuRecDial ). In DuRecDial , every dialog contains multi-type dialogs with natural topic transitions, which corresponds to the first difficulty. Moreover, there are rich interaction variability for recommendation, corresponding to the second difficulty. In addition, each seeker has an explicit profile for the modeling of personalized recommendation, and multiple dialogs with the recommender to mimic real-world application scenarios. To address this task, inspired by the work of Xu et al. (2020), we present a m ultig oal driven c onversation g eneration framework ( MGCG ) to handle multi-type dialogs simultaneously, such as QA/chitchat/recommendation/task etc.. It consists of a goal planning module and a goal-guided responding module. The goal-planning module can conduct dialog management to control the dialog flow, which determines a recommendation target as the final goal with consideration of user's interests and online feedback, and plans appropriate short-term goals for natural topic transitions. To our knowledge, this goal-driven dialog policy mechanism for multi-type dialog modeling is not studied in previous work. The responding module produces responses for completion of each goal, e.g., chatting about a topic or making a recommendation to the user. We conduct an empirical study of this framework on DuRecDial . This work makes the following contributions:  We identify the task of conversational recommendation over multi-type dialogs. To facilitate the study of this task, we create a novel dialog dataset DuRecDial , with rich variability of dialog types and domains as shown in Table 1.  We propose a conversation generation framework with a novel mixed-goal driven dialog policy mechanism. Datasets for Conversational Recommendation To facilitate the study of conversational recommendation, multiple datasets have been created in previous work, as shown in Table 1. The first recommendation dialog dataset is released by Dodge et al. (2016), which is a synthetic dialog dataset built with the use of the classic MovieLens ratings dataset and natural language templates. Li et al. (2018) creates a human-to-human multi-turn recommendation dialog dataset, which combines the elements of social chitchat and recommendation dialogs. Kang et al. (2019) provides a recommendation dialogue dataset with clear goals, and Moon et al. (2019) collects a parallel Dialog  KG corpus for recommendation. Compared with them, our dataset contains multiple dialog types, multi-domain use cases, and rich interaction variability. Datasets for Knowledge Grounded Conversation As shown in Table 1, CMU DoG (Zhou et al., 2018a) explores two scenarios for Wikipedia-article grounded dialogs: only one participant has access to the document, or both have. IIT DoG (Moghe et al., 2018) is another dialog dataset for movie chats, wherein only one participant has access to background knowledge, such as IMDB's facts/plots, or Reddit's comments. Dinan et al. (2019) creates a multi-domain multi-turn conversations grounded on Wikipedia articles. OpenDialKG (Moon et al., 2019) provides a chit-chat dataset between two agents, aimed at the modeling of dialog logic by walking over knowledge graph-Freebase. Wu et al. (2019) provides a Chinese dialog dataset-DuConv, where one participant can proactively lead the conversation with an explicit goal. KdConv (Zhou et al., 2020) is a Chinese dialog dataset, where each dialog contains in-depth discussions on multiple topics. In comparison with them, our dataset contains multiple dialog types, clear goals to achieve during each conversation, and user profiles for personalized conversation. Models for Conversational Recommendation Previous work on conversational recommender systems fall into two categories: (1) task-oriented dialog-modeling approaches in which the systems ask questions about user preference over prede-fined slots to select items for recommendation (Christakopoulou et al., 2018, 2016; Lee et al., 2018; Reschke et al., 2013; Sun and Zhang, 2018; Warnestal, 2005; Zhang et al., 2018b); (2) non-task dialog-modeling approaches in which the models learn dialog strategies from the dataset without pre-defined task slots and then make recommendations without slot filling (Chen et al., 2019; Kang et al., 2019; Li et al., 2018; Moon et al., 2019; Zhou et al., 2018a). Our work is more close to the second category, and differs from them in that we conduct multi-goal planning to make proactive conversational recommendation over multi-type dialogs. Goal Driven Open-domain Conversation Generation Recently, imposing goals on open-domain conversation generation models having attracted lots of research interests (Moon et al., 2019; Li et al., 2018; Tang et al., 2019b; Wu et al., 2019) since it provides more controllability to conversation generation, and enables many practical applications, e.g., recommendation of engaging entities. However, these models can just produce a dialog towards a single goal, instead of a goal sequence as done in this work. We notice that the model by Xu et al. (2020) can conduct multi-goal planning for conversation generation. But their goals are limited Ground-truthprofile of the seeker S k Seeker profile built so far P i-1Sk Task templates Knowledgegraph to in-depth chitchat about related topics, while our goals are not limited to in-depth chitchat. We define one person in the dialog as the recommendation seeker (the role of users) and the other as the recommender (the role of bots). We ask the recommender to proactively lead the dialog and then make recommendations with consideration of the seeker's interests, instead of the seeker to ask for recommendation from the recommender. Figure 2 shows our task design. The data collection consists of three steps: (1) collection of seeker profiles and knowledge graph; (2) collection of task templates; (3) annotation of dialog data. 3 Next we will provide details of each step. Explicit seeker profiles Each seeker is equipped with an explicit unique profile (a ground-truth profile), which contains the information of name, gender, age, residence city, occupation, and his/her preference on domains and entities. We automatically generate the ground-truth profile for each seeker, which is known to the seeker, and unknown to the recommender. We ask that the utterances of each seeker should be consistent with his/her profile. We expect that this setting could encourage the seeker to clearly and self-consistently explain what he/she likes/dislikes. In addition, the 2 Please see Appendix 1. for more details. recommender can acquire seeker profile information only through dialogs with the seekers. Knowledge graph Inspired by the work of document grounded conversation (Ghazvininejad et al., 2018; Moghe et al., 2018), we provide a knowledge graph to support the annotation of more informative dialogs. We build them by crawling data from Baidu Wiki and Douban websites. Table 3 presents the statistics of this knowledge graph. Multi-type dialogs for multiple domains We expect that the dialog between the two task-workers starts from a non-recommendation scenario, e.g., question answering or social chitchat, and the recommender should proactively and naturally guide the dialog to a recommendation target (an entity). The targets usually fall into the seeker's interests, e.g., the movies of the star that the seeker likes. Moreover, to be close to the setting in practical applications, we ask each seeker to conduct multiple sequential dialogs with the recommender. In the first dialog, the recommender asks questions about seeker profile. Then in each of the remaining dialogs, the recommender makes recommendations based on the seeker's preferences collected so far, and then the seeker profile is automatically updated at the end of each dialog. We ask that the change of seeker profile should be reflected in later dialogs. The difference between these dialogs lies in sub-dialog types and recommended entities. Rich variability of interaction How to iterate upon initial recommendation plays a key role in the interaction procedure for recommendation. To provide better supervision for this capability, we expect that the task workers can introduce diverse interaction behaviors in dialogs to better mimic the decision-making process of the seeker. For example, the seeker may reject the initial recommendation, or mention a new topic, or ask a question about an entity, or simply accept the recommendation. The recommender is required to respond appropriately and follow the seeker's new topic. Task templates as annotation guidance Due to the complexity of our task design, it is very hard to conduct data annotation with only high-level instructions mentioned above. Inspired by the work of MultiWOZ (Budzianowski et al., 2018), we provide a task template for each dialog to be annotated, which guides the workers to annotate in the way we expect them to be. As shown in Table 2, each template contains the following information: (1) a goal sequence, where each goal consists of two Goals Goal description Goal1: QA (dialog type) about the movie < Stolen life > (dialog topic) The seeker takes the initiative, and asks for the information about the movie < Stolen life > ; the recommender replies according to the given knowledge graph; finally the seeker provides feedback. elements, a dialog type and a dialog topic, corresponding to a sub-dialog. (2) a detailed description about each goal. We create these templates by (1) first automatically enumerating appropriate goal sequences that are consistent with the seeker's interests and have natural topic transitions and (2) then generating goal descriptions with the use of some rules and human annotation. To obtain this data, we develop an interface and a pairing mechanism. We pair up task workers and give each of them a role of seeker or recommender. Then the two workers conduct data annotation with the help of task templates, seeker profiles and knowledge graph. In addition, we ask that the goals in templates must be tagged in every dialog. Data structure We organize the dataset of DuRecDial according to seeker IDs. In DuRecDial , there are multiple seekers (each with a different profile) and only one recommender. Each seeker s k has multiple dialogs { d s k i } i with the recommender. For each dialog d s k i , we provide a knowledge graph and a goal sequence for data annotation, and a seeker profile updated with this dialog. Data statistics Table 3 provides statistics of knowledge graph and DuRecDial , indicating rich Knowledgegraph#Domains 7 #Entities 21,837 #Attributes 454 #Triples 222,198 DuRecDial #Dialogs 10,190 #Sub-dialogs for QA/Rec/task/chitchat 6,722/8,756/3,234/10,190 #Utterances 155,477 #Seekers 1362 #Entitiesrecom-mended/accepted/rejected11,162/8,692/2,470/ Table 3: Statistics of knowledge graph and DuRecDial. Data quality We conduct human evaluations for data quality. A dialog will be rated 1 if it follows the instruction in task templates and the utterances are fluent and grammatical, otherwise 0. Then we ask three persons to judge the quality of 200 randomly sampled dialogs. Finally we obtain an average score of 0.89 on this evaluation set. Problem definition Let D s k = { d s k i } N Dsk i =0 denote a set of dialogs by the seeker s k (0  k < N s ) , where ND sk is the number of dialogs by the seeker s k , and N s is the number of seekers. Recall that we attach each dialog (say d s k i ) with an updated seeker profile (denoted as P s k i ), a knowledge graph K , and a goal sequence G = { g t } T g  1 t =0 . Given a context X with utterances { u j } m  1 j =0 from the dialog d s k i , a goal history G (cid:48) = { g 0 , ..., g t  1 } ( g t  1 as the goal for u m  1 ), P s k i  1 and K , the aim is to provide an appropriate goal g c to determine where the dialog goes and then produce a proper response Y = { y 0 , y 1 , ..., y n } for completion of the goal g c . Framework overview The overview of our framework MGCG is shown in Figure 3. The goal-planning module outputs goals to proactively and naturally lead the conversation. It first takes as input X , G (cid:48) , K and P s k i  1 , then outputs g c . The responding module is responsible for completion of each goal by producing responses conditioned on X , g c , and K . For implementation of the responding module, we adopt a retrieval model and a generation model proposed by Wu et al. (2019), and modify them to suit our task. For model training, each [context, response] in d s k i is paired with its ground-truth goal, P s k i and K . These goals will be used as answers for training of the goal-planning module, while the tuples of [context, a ground-truth goal, K , response] will be used for training of the responding module. As shown in Figure 3(a), we divide the task of goal planning into two sub-tasks, goal completion estimation, and current goal prediction. Goal completion estimation For this subtask, we use Convolutional neural network (CNN)(Kim, 2014) to estimate the probability of goal completion by: PGC ( l = 1 | X, g t  1 ) . (1) Current goal prediction If g t  1 is not completed ( PGC < 0 . 5 ), then g c = g t  1 , where g c is the goal for Y . Otherwise, we use CNN based multi-task classification to predict the current goal by maximizing the following probability: g t = arg max g ty ,g tp PGP ( g ty , g tp | X, G (cid:48) , P s k i , K ) , (2) g c = g t , (3) where g ty is a candidate dialog type and g tp is a candidate dialog topic. we modify the original retrieval model to suit our task by emphasizing the use of goals. As shown in Figure 3(b), our response ranker consists of five components: a context-response representation module (C-R Encoder), a knowledge representation module (Knowledge Encoder), a goal representation module (Goal Encoder), a knowledge selection module (Knowledge Selec-tor), and a matching module (Matcher). The C-R Encoder has the same architecture as BERT (Devlin et al., 2018), and it takes a context X and a candidate response Y as segment a and segment b in BERT, and leverages a stacked self-attention to produce the joint representation of X and Y , denoted as xy . Each related knowledge knowledge i is also encoded as a vector by the Knowledge Encoder using a bi-directional GRU (Chung et al., 2014), which can be formulated as k i = [  h T k ;  h 0 ] , where T k denotes the length of knowledge,  h T k and  h 0 represent the last and initial hidden states of the two directional GRU respectively. The Goal Encoder uses bi-directional GRUs to encode a dialog type and a dialog topic for goal representation (denoted as g c ). For knowledge selection, we make the context-response representation xy attended to all knowledge vectors k i and get the attention distribution: p ( k i | x, y, g c ) = exp ( MLP ([ xy ; g c ])  k i ) (cid:80) j exp ( MLP ([ xy ; g c ])  k j ) (4) We fuse all related knowledge information into a single vector k c = (cid:80) i p ( k i | x, y, g c )  k i . We view k c , g c and xy as the information from knowledge source, goal source and dialogue source respectively, and fuse the three information sources into a single vector via concatenation. Finally we calculate a matching probability for each Y by: p ( l = 1 | X, Y, K, g c ) = softmax ( MLP ([ xy ; k c ; g c ])) (5) 4.4 Generation-based Response Model To highlight the importance of conversational goals, we also modify the original generation model by introducing an independent encoder for goal representation. As shown in Figure 3(c), our generator consists of five components: a Context Encoder, a Knowledge Encoder, a Goal Encoder, a Knowledge Selector, and a Decoder. Given a context X , conversational goal g c and knowledge graph K , our generator first encodes them as vectors with the use of above encoders (based on bi-directional GRUs). We assume that using the correct response will be conducive to knowledge selection. Then minimizing KLDivloss will make the effect of knowledge selection in the prediction stage (not use the correct response) close to that of knowledge selection with correct response. For knowledge selection, the model learns knowledge-selection strategy through minimizing the KLDivLoss between two distributions, a prior distribution p ( k i | x, g c ) and a posterior distribution p ( k i | x, y, g c ) . It is formulated as: p ( k i | x, y, g c ) = exp ( k i  MLP ([ x ; y ; g c ])) (cid:80) N j =1 exp ( k j  MLP ([ x ; y ; g c ])) (6) p ( k i | x, g c ) = exp ( k i  MLP ([ x ; g c ]) (cid:80) N j =1 exp ( k j  MLP ([ x ; g c ]) (7) LKL (  ) = 1 NN (cid:88) i =1 p ( k i | x, y, g c ) log p ( k i | x, y, g c ) p ( k i | x, g c ) (8) In training procedure, we fuse all related knowledge information into a vector k c = (cid:80) i p ( k i | x, y, g c )  k i , same as the retrieval-based method, and feed it to the decoder for response generation. In testing procedure, the fused knowledge is estimated by k c = (cid:80) i p ( k i | x, g c )  k i without ground-truth responses. The decoder is implemented with the Hierarchical Gated Fusion Unit described in (Yao et al., 2017), which is a standard GRU based decoder enhanced with external knowledge gates. In addition to the loss LKL (  ) , the generator uses the following losses: NLL Loss : It computes the negative log-likelihood of the ground-truth response ( LNLL (  ) ). BOW Loss : We use the BOW loss proposed by Zhao et al. (2017), to ensure the accuracy of the fused knowledge k c by enforcing the relevancy between the knowledge and the true response. 4 Specifically, let w = MLP ( k c )  R | V | , where | V | is vocabulary size. We define: p ( y t | k c ) = exp ( w y t ) (cid:80) V v =1 exp ( w v ) . Then, the BOW loss is defined to minimize: Finally, we minimize the following loss function: L (  ) =   LKL (  ) +   LNLL (  ) + LBOW (  ) (11) where  is a trainable parameter. We split DuRecDial into train/dev/test data by randomly sampling 65%/10%/25% data at the level of seekers, instead of individual dialogs. To evaluate the contribution of goals, we conduct an ablation study by replacing input goals with UNK for responding model. For knowledge usage, we conduct another ablation study, where we remove input knowledge by replacing them with UNK. S2S We implement a vanilla sequence-to-sequence model (Sutskever et al., 2014), which is widely used for open-domain conversation generation. MGCG R : Our system with automatic goal planning and a retrieval based responding model. MGCG G : Our system with automatic goal planning and a generation based responding model. 4 The BOW loss is to introduce an auxiliary loss that requires the decoder network to predict the bag-of-words in the response to tackle the vanishing latent variable problem. 5 Please see Appendix 2. for model parameter settings. Metrics For automatic evaluation, we use several common metrics such as BLEU (Papineni et al., 2002), F1, perplexity (PPL), and DISTINCT (DIST-2) (Li et al., 2016) to measure the relevance, fluency, and diversity of generated responses. Following the setting in previous work (Wu et al., 2019; Zhang et al., 2018a), we also measure the performance of all models using Hits@1 and Hits@3. 6 Here we let each model to select the best response from 10 candidates. Those 10 candidate responses consist of the ground-truth response generated by humans and nine randomly sampled ones from the training set. Moreover, we also evaluate the knowledge-selection capability of each model by calculating knowledge precision/recall/F1 scores as done in Wu et al. (2019). 7 In addition, we also report the performance of our goal planning module, including the accuracy of goal completion estimation, dialog type prediction, and dialog topic prediction. Results Our goal planning model can achieve accuracy scores of 94.13%, 91.22%, and 42.31% for goal completion estimation, dialog type prediction, and dialog topic prediction. The accuracy of dialog topic prediction is relatively low since the num-6 Candidates (including golden response) are scored by PPL using the generation-based model, then candidates are sorted based on the scores, and Hits@1 and Hits@3 are calculated. ber of topic candidates is very large (around 1000), leading to the difficulty of topic prediction. As shown in Table 4, for response generation, both MGCG R and MGCG G outperform S2S by a large margin in terms of all the metrics under the same model setting (without gl.+kg., with gl., or with gl.+kg.). Moreover, MGCG R performs better in terms of Hits@k and DIST-2, but worse in terms of knowledge F1 when compared to MGCG G. 8 It might be explained by that they are optimized on different metrics. We also found that the methods using goals and knowledge outperform those without goals and knowledge, confirming the benefits of goals and knowledge as guidance information. Metrics: The human evaluation is conducted at the level of both turns and dialogs. For turn-level human evaluation, we ask each model to produce a response conditioned on a given context, the predicted goal and related knowledge. 9 The generated responses are evaluated by three annotators in terms of fluency, appropriateness, informativeness, and proactivity. The appropriateness measures if the response can complete current goal and it is also relevant to the context. The informativeness measures if the model makes full use of knowledge in the response. The proactivity mea-8 We calculate an average of F1 over all the dialogs. It might result in that the value of F1 is not between P and R. 9 Please see Appendix 3. for more details. sures if the model can successfully introduce new topics with good fluency and coherence. For dialogue-level human evaluation, we let each model converse with a human and proactively make recommendations when given the predicted goals and related knowledge. 10 For each model, we collect 100 dialogs. These dialogs are then evaluated by three persons in terms of two metrics: (1) goal success rate that measures how well the conversation goal is achieved, and (2) coherence that measures relevance and fluency of a dialog as a whole. All the metrics has three grades: good(2), fair(1), bad(0). For proactivity, 2 indicates that the model introduces new topics relevant to the context, 1 means that no new topics are introduced, but knowledge is used, 0 means that the model introduces new but irrelevant topics. For goal success rate, 2 means that the system can complete more than half of the goals from goal planning module, 0 means the system can complete no more than one goal, otherwise 1. For coherence, 2/1/0 means that two-thirds/one-third/very few utterance pairs are coherent and fluent. Results All human evaluations are conducted by three persons. As shown in Table 5, our two systems outperform S2S by a large margin, especially in terms of appropriateness, informativeness, goal success rate and coherence. In particular, S2S tends to generate safe and uninformative responses, failing to complete goals in most of dialogs. Our two systems can produce more appropriate and informative responses to achieve higher goal success rate with the full use of goal information and knowledge. Moreover, the retrieval-based model performs better in terms of fluency since its response is selected from the original human utterances, not automatically generated. But it performs worse on all the other metrics when compared to the generation-based model. It might be caused by the limited number of retrieval candidates. Finally, it can be seen that there is still much room for performance improvement in terms of appropriateness and goal success rate, which will be left as the future work. In order to further analyze the relationship between knowledge usage and goal completion, we provide the number of failed goals, completed goals, and used knowledge for each method over different dialog types in Table 6. We see that the number of 10 Please see Appendix 4. for more details. used knowledge is proportional to goal success rate across different dialog types or different methods, indicating that the knowledge selection capability is crucial to goal completion through dialogs. Moreover, the goal of chitchat dialog is easier to complete in comparison with others, and QA and recommendation dialogs are more challenging to complete. How to strengthen knowledge selection capability in the context of multi-type dialogs, especially for QA and recommendation, is very important, which will be left as the future work. We identify the task of conversational recommendation over multi-type dialogs, and create a dataset DuRecDial with multiple dialog types and multi-domain use cases. We demonstrate usability of this dataset and provide results of state of the art models for future studies. The complexity in DuRecDial makes it a great testbed for more tasks such as knowledge grounded conversation (Ghazvininejad et al., 2018), domain transfer for dialog modeling, target-guided conversation (Tang et al., 2019a) and multi-type dialog modeling (Yu et al., 2017). The study of these tasks will be left as the future work. We would like to thank Ying Chen for dataset annotation and thank Yuqing Guo and the reviewers for their insightful comments. This work was supported by the National Key Research and Development Project of China (No. 2018AAA0101900) and the Natural Science Foundation of China (No. 61976072).
method method abstain abstain result abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain objective abstain abstain abstain objective other abstain abstain method abstain abstain abstain method abstain abstain objective abstain method objective objective other other other method other other other other other other objective other objective other objective abstain other objective method abstain abstain abstain method abstain method abstain method abstain abstain method method abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain method method other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain abstain method other method abstain abstain method abstain abstain abstain abstain method method abstain method method abstain method abstain other abstain method method method abstain method method method abstain abstain method abstain abstain abstain method abstain method abstain method abstain abstain abstain abstain abstain method abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain method objective abstain abstain other other