Recently, various response generation models for two-party conversations have achieved impressive improvements, but less effort has been paid to multi-party conversations (MPCs) which are more practical and complicated. Compared with a two-party conversation where a dialogue context is a sequence of utterances, building a response generation model for MPCs is more challenging, since there exist complicated context structures and the generated responses heavily rely on both interlocutors (i.e., speaker and addressee) and history utterances. To address these challenges, we present HeterMPC, a heterogeneous graph-based neural network for response generation in MPCs which models the semantics of utterances and interlocutors simultaneously with two types of nodes in a graph. Besides, we also design six types of meta relations with node-edge-type-dependent parameters to characterize the heterogeneous interactions within the graph. Through multi-hop updating, HeterMPC can adequately utilize the structural knowledge of conversations for response generation. Experimental results on the Ubuntu Internet Relay Chat (IRC) channel benchmark show that HeterMPC outperforms various baseline models for response generation in MPCs. Enabling dialogue systems to converse naturally with humans is a challenging yet intriguing problem of artificial intelligence and has attracted increasing attention due to its promising potentials and alluring commercial values (Kepuska and Bohouta, 2018; Berdasco et al., 2019; Zhou et al., 2020). A large number of researchers have focused on building dialogue generation models with various neural networks. At first, researchers mostly  Work done during the internship at Microsoft. (b) Graphical information flow in multi-party conversation Figure 1: Illustration of a graphical information flow in an MPC. Pink rectangles denote utterances and blue circles denote interlocutors. Each solid line represents the  replied-by " relationship between two utterances. Each dashed line indicates the speaker of an utterance. focused on dialogues between two participants (Shang et al., 2015; Serban et al., 2016; Wen et al., 2017; Young et al., 2018). Recently, researchers have paid more attention to a more practical and challenging scenario involving more than two participants, which is well known as multi-party conversations (MPCs) (Ouchi and Tsuboi, 2016; Zhang et al., 2018; Le et al., 2019; Hu et al., 2019b; Wang et al., 2020b; Gu et al., 2021). Utterances in a two-party conversation are posted one by one between two interlocutors, constituting a sequential information flow. Different from that, utterances in an MPC can be spoken by anyone and address anyone else in this conversation, which constitutes a graphical information flow as shown in Figure 1. Although sequence-to-sequence (Seq2Seq) models (Sutskever et al., 2014; Serban et al., 2016) are effective at modeling sequential dialogues, they fall short of modeling graph-structured ones. To overcome this drawback, Hu et al. (2019b) first proposed a graph-structured network (GSN) to encode utterances based on the graph topology rather than the sequence of their appearances. The graph established in GSN was homogeneous, where nodes represented only utterances. However, interlocutors are also important components of MPCs. There exist complicated interactions between interlocutors, and between an utterance 5086 and an interlocutor. Furthermore, when passing messages over a graph, a bidirectional information flow algorithm was designed for GSN. Since both the forward and backward information flows employed the same model structure and parameters, this algorithm cannot distinguish the  reply " or  replied-by " relations between two connected utterance nodes. Also, information flows along both directions are independently propagated, so that a graph node cannot be jointly updated at a single propagation step. On account of above issues, we propose a heterogeneous graph-based neural network for response generation in MPCs, named HeterMPC. First, a heterogeneous graph is designed which employs two types of nodes to represent utterances and interlocutors respectively. Different from previous methods that built a homogeneous graph modeling only utterances, utterances and interlocutors are modeled simultaneously in HeterMPC, so that the complicated interactions between interlocutors, between utterances, and between an interlocutor and an utterance can be explicitly described. In order to characterize the heterogeneous attention over each ( source, edge, target ) triple, model parameters dependent on both types of nodes and edges are introduced when calculating attention weights and passing messages. Specifically, we introduce six types of meta relations for modeling different edges including  reply  and  replied-by  between two utterances,  speak  and  spoken-by  between an utterance and a speaker, and  address  and  addressed-by  between an utterance and an addressee. With these node-edge-type-dependent structures and parameters, HeterMPC can better utilize the structural knowledge of conversations for node representation and response generation than conventional homogeneous graphs. Finally, Transformer is employed as the backbone of HeterMPC and its model parameters can be initialized with PLMs to take advantage of the recent breakthrough on pre-training. We evaluate HeterMPC on the Ubuntu Internet Relay Chat (IRC) channel benchmark released by Hu et al. (2019b). Experimental results show that HeterMPC outperforms GSN (Hu et al., 2019b), GPT-2 (Radford et al., 2019), BERT (Devlin et al., 2019) and BART (Lewis et al., 2020) by significant margins in terms of both automated and human evaluation metrics, achieving a new state-of-the-art performance for response generation in MPCs. In summary, our contributions in this paper are three-fold: 1) To the best of our knowledge, this paper is the first exploration of using heterogeneous graphs for modeling conversations; 2) A Transformer-based heterogeneous graph architecture is introduced for response generation in MPCs, in which two types of nodes, six types of meta relations, and node-edge-type-dependent parameters are employed to characterize the heterogeneous properties of MPCs; 3) Experimental results show that our proposed model achieves a new state-of-the-art performance of response generation in MPCs on the Ubuntu IRC benchmark. 2 Related Work Multi-Party Conversation Existing methods on building dialogue systems can be generally categorized into generation-based (Shang et al., 2015; Serban et al., 2016; Wen et al., 2017; Young et al., 2018; Zhang et al., 2020) or retrieval-based approaches (Lowe et al., 2015; Wu et al., 2017; Zhou et al., 2018; Tao et al., 2019a,b; Gu et al., 2019, 2020). In this paper, we study the task of response generation in MPCs, where in addition to utterances, interlocutors are also important components who play the roles of speakers or addressees. Previous methods have explored retrieval-based approaches for MPCs. For example, Ouchi and Tsuboi (2016) proposed the dynamic model which updated speaker embeddings with conversation streams. Zhang et al. (2018) proposed speaker interaction RNN which updated speaker embeddings role-sensitively. Wang et al. (2020b) proposed to track the dynamic topic in a conversation. Gu et al. (2021) proposed jointly learning who says what to whom" in a unified framework by designing self-supervised tasks during pre-training. On the other hand, Hu et al. (2019b) explored generation-based approaches by proposing a graph-structured network, the core of which was an utterance-level graph-structured encoder. Heterogeneous Graph Neural Network Early studies on graph neural networks (GNNs) focused on homogeneous graphs where a whole graph is composed of a single type of nodes. However, graphs in real-world applications usually come with multiple types of nodes, namely heterogeneous information networks (HINs) or heterogeneous graphs (Sun and Han, 2012). Recently, researchers have attempted to extend GNNs to model heterogeneity. For example, Zhang et al. 5087 (2019) adopted different RNNs for different types of nodes to integrate multi-modal features. Wang et al. (2019) extended graph attention networks by maintaining different weights for different meta-path-defined edges. Hu et al. (2020) proposed heterogeneous graph Transformer (HGT) to model heterogeneity by maintaining dedicated representations for different types of nodes and edges. In addition, heterogeneous graphs have also been applied to many NLP tasks, such as multi-hop reading comprehension (Tu et al., 2019), text classification (Hu et al., 2019a) and document summarization (Wang et al., 2020a). Previous studies have verified the superiority of modeling MPCs with homogeneous graphs considering only utterances. We claim that it is indeed necessary to model a complex information flow in MPCs shown in Figure 1 with a heterogeneous graph, since a homogeneous one cannot explicitly model the relationships of multiple utterances spoken by or addressing an interlocutor. Nowadays, HINs have been widely used in many NLP tasks. To the best of our knowledge, this paper makes the first attempt to build a heterogeneous graph-based neural network considering utterances and interlocutors simultaneously for response generation in MPCs. In addition, we introduce many task-specific modelings for MPCs such as graph construction and node updating which will be elaborated in the model section. The task of response generation in MPCs is to generate an appropriate response  r given the conversation history, the speaker of a response, and which utterance the response is going to reply to, which can be formulated as: Here, G is a heterogeneous graph containing both history conversation and the response to be generated. The speaker and addressee of the response are known and its contents are masked. The response tokens are generated in an autoregressive way. r k and r <k stand for the k -th token and the first ( k  1) tokens of response r respectively. | r | is the length of r . We will introduce how to construct the graph and how to model the probability in Eq. (1) given the built graph in the next section. HeterMPC adopts an encoder-decoder architecture consisting of stacked encoder and decoder layers for graph-to-sequence learning (Yao et al., 2020). The graph encoder is designed to capture conversation structures and output the representations of all nodes in a graph that are fed to the decoder for response generation. A heterogeneous graph is constructed to explicitly model the complicated interactions between interlocutors, between utterances, and between an interlocutor and an utterance in an MPC. This graph models utterances and interlocutors by considering them as two types of nodes under a unified framework. Given an MPC instance composed of M utterances and I interlocutors, a heterogeneous graph G ( V , E ) is constructed. Specifically, V is a set of M + I nodes. Each node denotes either an utterance or an interlocutor. E = { e p,q } M + I p,q =1 is a set of directed edges. Each edge e p,q describes the connection from node p to node q . Inspired by Sun et al. (2011, 2012), we introduce six types of meta relations { reply , replied-by , speak , spoken-by , address , addressed-by } to describe the directed edge between two graph nodes as illustrated in Figure 2. Specifically, if an utterance represented by node n replies another utterance represented by node m , the edge e n,m = reply and the reversed edge e m,n = replied-by . If an utterance represented by node m is spoken by an interlocutor represented by node i , e i,m = speak and e m,i = spoken-by . If an utterance represented by node n addresses an interlocutor represented by 5088 Transformer L 1 X L 2 X Transformer Transformer ATT r e p l y WMSG r e p l y WATT r e p li e d b y W  MSG r e p li e d b y W  ATT s p e a k WMSG s p e a k WATT add r e ss e d b y W  MSG add r e ss e d b y W  FFNUTR Transformer  QUTRWKUTRWVUTRWKUTRWVUTRWKITRWVITRWKITRWVITRW Target node Source nodes  Transformer Transformer ATT s po ke n b y W  MSG s po ke n b y W  ATT add r e ss WMSG add r e ss WKUTRWVUTRWKUTRWVUTRWQITRWFFNITR Source nodes Target node node i , e n,i = address and e i,n = addressed-by . In other cases, e p,q = NULL indicating that there is no connection between these two nodes. Note that it is necessary to distinguish the bidirectional edges between every two nodes that indicate the active and passive tense information respectively. In HeterMPC, each node is represented as a vector. These vectors are first initialized individually without considering graph edges. Utterances When encoding utterances, a [CLS] token is inserted at the start of each utterance, denoting the utterance-level representation. Besides, a [SEP] token is also inserted at the end of each utterance (Devlin et al., 2019). Then each utterance is encoded individually by stacked Transformer encoder layers through the self-attention mechanism to derive the contextualized utterance representations. 1 The output of a Transformer encoder layer is used as the input of the next layer. Readers can refer to Vaswani et al. (2017) for details of Transformer. Formally, the calculation for an utterance at the l -th Transformer layer is denoted as: H l +1 m = TransformerEncoder( H lm ) , (2) 1 In our experiments, BERT or BART was selected to initialize the utterance encoder layers of HeterMPC. Then, the built HeterMPC models were compared with the baseline models directly finetuning BERT or BART, respectively. It is worth noting that the utterance encoder layers of HeterMPC can also be initialized by other types of PLMs, and the comparison across PLMs is not the focus of this paper. where m  { 1 , ..., M } , l  { 0 , ..., L 1  1 } , L 1 denotes the number of Transformer layers for initialization, H lm  R k m  d , k m denotes the length of an utterance and d denotes the dimension of embedding vectors. Interlocutors Different from an utterance composed of a sequence of tokens, an interlocutor is directly represented with an embedding vector. Interlocutors in a conversation are indexed according to their speaking order and the embedding vector for each interlocutor is derived by looking up an order-based interlocutor embedding table (Gu et al., 2020) that is updated during end-to-end learning. The first interlocutors in all conversation sessions share the same embedding vector in the interlocutor embedding table, so do all the second interlocutors. 2 Thus, this order-based embedding table can be shared across the training, validation and testing sets, and there is no need to estimate an embedding vector for each specific interlocutor in the dataset. As shown in Figure 3, the initialized node representations are updated by feeding them into the built graph for absorbing context information (Kipf and Welling, 2017; Velickovic et al., 2018; Yun 2 In our experiments, the maximum interlocutor number was set to 10 and an embedding table sized 10*768 was learned during training. We did study initializing the embedding vector of an interlocutor node by averaging the representations of all utterance nodes it speaks, but no further improvement can be achieved. et al., 2019). We calculate heterogeneous attention weights between connected nodes and pass messages over the graph in a node-edge-type-dependent manner, inspired by introducing parameters to maximize feature distribution differences for modeling heterogeneity (Schlichtkrull et al., 2018; Wang et al., 2019; Zhang et al., 2019; Hu et al., 2020). After collecting the information from all source nodes to a target node, a node-type-dependent feed-forward network (FFN) followed by a residual connection (He et al., 2016) is employed to aggregate the information. Then, in order to let each token in an utterance have access to the information from other utterances, an additional Transformer layer is placed for utterance nodes specifically. L 2 denotes the number of iterations for updating both utterance and interlocutor nodes. Since the representations of two types of nodes are initialized in different ways, node-type-dependent linear transformations are first applied to node representations before attention calculation so that the two types of nodes share similar feature distributions (Wang et al., 2019; Hu et al., 2020). Meanwhile, each of the six relation types is assigned a separate linear projection so that the semantic relationship between two connected nodes can be accurately described when calculating attention weights. The forward and backward information flows between them can also be distinguished. Formally, let the triple ( s, e, t ) denote an edge e connecting a source node s to a target node t . The representations of the source and target nodes at the l -th iteration 3 are denoted as h ls and h lt , serving as a key ( K ) vector and a query ( Q ) vector of attention calculation respectively. Then, the heterogeneous attention weight w l ( s, e, t ) before normalization for this triple is calculated as: k l ( s ) = h ls W K ( s ) + b K ( s ) , (3) q l ( t ) = h lt W Q ( t ) + b Q ( t ) , (4) w l ( s, e, t ) = k l ( s ) W ATTe s,t q l ( t ) T  e s,t  d . (5) Here,  ( s ) ,  ( t )  {UTR, ITR} distinguish utterance ( UTR ) and interlocutor ( ITR ) nodes. Eqs. (3) and (4) are node-type-dependent linear transformations. Eq. (5) contains an edge-type-dependent linear projection W ATTe s,t where  e s,t is an adaptive 3 For an utterance, the representation for the [CLS] token is extracted as the utterance-level representation. When passing the message of a source node that serves as a value ( V ) vector to a target node, node-edge-type-dependent parameters are also introduced considering the heterogeneous properties of nodes and edges. Mathematically: v l ( s ) = h ls W V ( s ) + b V ( s ) , (6)  v l ( s ) = v l ( s ) W MSGe s,t , (7) where  v l ( s ) is the passed message and all W   R d  d and b   R d are parameters to be learnt. For a target node, the messages passed from all its connected source nodes need to be aggregated. A softmax function is applied to normalize the attention weights and then the messages from all source codes are summarized as:  h lt = (cid:88) s  S ( t ) softmax( w l ( s, e, t ))  v l ( s ) , (8) where S ( t ) denotes the set of source nodes for the target node t . Then the summarized message  h lt is aggregated with the original node representation h lt using a node-type-dependent FFN followed by a residual connection (He et al., 2016) as: h l +1 t = FFN  ( t ) (  h lt ) + h lt , (9) where the output h l +1 t is used as the input of the next iteration of node updating. One iteration can be viewed as a single-step information propagation along edges. When stacking L 2 iterations, a node can attend to other nodes up to L 2 hops away. A specific consideration on utterance nodes is that the tokens except [CLS] in an utterance have no access to other utterances during the node updating process introduced above. To overcome this disadvantage and derive more contextualized utterance representations, an additional Transformer layer (Vaswani et al., 2017) is further placed for utterance nodes as shown in Figure 3. In detail, at the l -th iteration, the representations of an utterance node before and after node updating, i.e., h lt and h l +1 t , are concatenated and then compressed by a linear transformation as:  h l +1 t = [ h lt ; h l +1 t ] W com + b com , (10) 5090 Masked Self Attention Add & Norm Cross Attention Add & Norm Feed Forward Add & Norm Linear Softmax Output Probabilities Graph Encoder L 3 x Input: Context Output: Response Node Representations Figure 4: The decoder architecture of HeterMPC. where W com  R 2 d  d and b com  R d are parameters. Then,  h l +1 t replaces the representation of [CLS] (i.e., h lt ) in the sequence representations of the whole utterance. Finally, the updated sequence representations are fed into the additional Transformer layer for another round of intra-utterance self-attention, so that the context information learnt by the [CLS] representation can be shared with other tokens in the utterance. The decoder is composed of a stack of identical layers as shown in Figure 4. We follow the standard implementation of Transformer decoder to generate responses. In each decoder layer, a masked self-attention operation is first performed where each token cannot attend to future tokens to avoid information leakage. Furthermore, a cross-attention operation over the node representations of the graph encoder output is performed to incorporate graph information for decoding. It is notable that a residual connection along with layer normalization is followed by each attention operation. We evaluated our proposed method on the Ubuntu IRC benchmark used in Hu et al. (2019b). The data processing script provided by Hu et al. (2019b) was employed to derive the dataset. 4 In this dataset, both speaker and addressee labels were included for each utterance in a session. When testing, the 4 We contacted the authors of Hu et al. (2019b) to obtain the data processing script. As they claimed, it was an updated version which was a little different from that used in their paper. Thus, we re-implemented all baselines on this updated dataset to ensure fair comparison. speaker and addressee information was both given for response generation, i.e., the system knew who would speak next and which utterance should be responded to following the graph structure. It contained 311,725/5,000/5,000 dialogues in the training/validation/testing sets respectively. We compared our proposed methods with as many MPC models as possible. Considering that there are only a few research papers in this field, several recent advanced models were also adapted to provide sufficient comparisons. Finally, we compared with the following baseline models: (1) RNN-based Seq2Seq (Sutskever et al., 2014) took all utterances except the target utterance to generate as input, which were sorted according to their posting time and concatenated. Thus, structured conversations were converted into sequential ones. Seq2Seq modeling with attention was performed as that in Sutskever et al. (2014); Bahdanau et al. (2015) on the concatenated utterances. (2) Transformer (Vaswani et al., 2017) took the same input utterances as those used for the Seq2Seq model. (3) GPT-2 (Radford et al., 2019) was a uni-directional pre-trained language model. Following its original concatenation operation, all context utterances and the response were concatenated with a special [SEP] token as input for encoding. (4) BERT (Devlin et al., 2019) concatenated all context utterances and the response similarly as those for GPT-2. To adapt BERT for response generation, a special masking mechanism was designed to avoid response information leakage during encoding. Concretely, each token in the context utterances attended to all tokens in the context utterances, while each token in the response cannot attend to future tokens in the utterance. (5) GSN (Hu et al., 2019b) achieved the state-of-the-art performance on MPCs. The core of GSN was an utterance-level graph-structured encoder. (6) BART (Lewis et al., 2020) was a denoising autoencoder using a standard Tranformer-based architecture, trained by corrupting text with an arbitrary noising function and learning to reconstruct the original text. In our experiments, a concatenated context started with <s> and separated with </s> were fed into the encoder, and a response were fed into the decoder. To ensure all experimental results were comparable, we used the same automated and human evaluation metrics as those used in previous work (Hu et al., 2019b). Hu et al. (2019b) used the evaluation package released by Chen et al. (2015) including BLEU-1 to BLEU-4, METEOR and ROUGEL , which was also used in this paper. 5 Human evaluation was conducted to measure the quality of the generated responses in terms of three independent aspects: 1) relevance, 2) fluency and 3) informativeness. Each judge was asked to give three binary scores for a response, which were further summed up to derive the final score ranging from 0 to 3. 5.4 Training Details Model parameters were initialized with pre-trained weights of bert-base-uncased released by Wolf et al. (2020). The AdamW method (Loshchilov 5 https://github.com/tylin/coco-caption and Hutter, 2019) was employed for optimization. The learning rate was initialized as 6 . 25 e 5 and was decayed linearly down to 0 . The max gradient norm was clipped down to 1 . 0 . The batch size was set to 16 with 8 gradient accumulation steps. The maximum utterance length was set to 50 . The number of layers for initializing utterance representations L 1 was set to 9, and the number of layers for heterogeneous graph iteration L 2 was set to 3. L 1 and L 2 were validated on the validation set. The number of decoder layers L 3 was set to 6, achieving the best performance out of {2, 4, 6, 8} on the validation set. The strategy of greedy search was performed for decoding. The maximum length of responses for generation was also set to 50 . All experiments were run on a single GeForce RTX 2080 Ti GPU. The maximum number of epochs was set to 15, taking about 40 hours. The validation set was used to select the best model for testing. All code was implemented in the PyTorch framework 6 and are published to help replicate our results. 7 5.5 Evaluation Results In our experiments, BERT and BART were selected to initialize HeterMPC. HeterMPC BERT denoted that the utterance encoder was initialized with BERT and the decoder was randomly initialized. HeterMPC BART denoted the encoder and decoder 6 https://pytorch.org/ 7 https://github.com/lxchtan/HeterMPC 5092 were initialized by those of BART respectively. Automated Evaluation Table 1 presents the evaluation results of HeterMPC BERT , HeterMPC BART and previous methods on the test set. Each model ran four times with identical architectures and different random initializations, and the best out of them was reported. We ran the code released by Hu et al. (2019b) to reproduce the results of GSN for a fair comparison. 8 The results show that both HeterMPC BERT and HeterMPC BART outperformed all baselines in terms of all metrics. HeterMPC BERT outperformed GSN by 2.38% BLEU-1 and 0.44% BLEU-4, and outperformed GPT-2 by 2.24% BLEU-1 and 0.48% BLEU-4. HeterMPC BART outperformed GSN by 2.03% BLEU-1 and 0.52% BLEU-4, and outperformed GPT-2 by 1.89% BLEU-1 and 0.56% BLEU-4. Furthermore, HeterMPC BERT outperformed BERT by 1.71% BLEU-1 and 0.52% BLEU-4, and HeterMPC BART outperformed BART by 1.01% BLEU-1 and 0.54% BLEU-4, illustrating the importance of modeling MPC structures. To further verify the effectiveness of our proposed methods, ablation tests were conducted as shown in Table 1. First, all nodes or edges were considered equivalently by employing the same linear transformations in Eqs. (3) to (9) for all node or edge types without distinguishing them. The drop in performance illustrates the effectiveness of the node-edge-type-dependent parameters. On the other hand, interlocutor nodes were removed out of a graph and only the meta relations of reply and replied-by were left. The drop in performance illustrates the importance of modeling interactions between utterances and interlocutors, and the effectiveness of the heterogeneous architecture. Human Evaluation Table 2 presents the human evaluation results on a randomly sampled test set. 200 samples were evaluated and the order of evaluation systems were shuffled. Three graduate students were asked to score from 0 to 3 (3 for the best) and the average scores were reported. The Fleiss's kappa value (Fleiss, 1971) for each model was also reported, indicating the inter-judge moderate agreement during evaluation. It can be seen that HeterMPC BERT and HeterMPC BART achieved higher subjective quality scores than the baselines. Their kappa values were also higher than the BERT and BART baselines, respectively. The impact of numbers of iterations ( L 2 ). Figure 5 illustrates how the performance of HeterMPC BERT changed with respect to different numbers of iterations ( L 2 ) on the test set. It can be seen that the performance of HeterMPC BERT was significantly improved as L 2 increased at the beginning, which shows the effectiveness of incorporating the contextual information between nodes. Then, the performance was stable and dropped slightly. The reason might be that models begin to overfit due to a larger set of parameters. The impact of conversation length. 9 Figure 6 illustrates how the performance of HeterMPC BERT changed according to the test samples with different session lengths. As the session length increased, the performance of HeterMPC BERT dropped less than that of BERT, showing superiority of our method on dealing with longer conversations. Case Study. Case studies were conducted by randomly sampling two MPC instances as shown in Table 3. Given the conversation graph of the first case, the response to generate addresses I.2. Thus, the information relevant to I.2 should be collected. We can see that  gparted  in the first utterance is two hops away from I.2 (the first utterance is replied by the second utterance which is spoken by I.2), and this word in the fourth utterance and  install gparted  in the third utterance are both one hop away from I.2 (these two utterances directly address I.2). The responses generated by HeterMPC BERT and HeterMPC BART both contain these keywords, showing that it can capture the conversation graph information accurately and generate a human-like response. However, due to the lack of the interlocutor information and the conversation structure, GSN generated an irrelevant response. BERT generated a response which seems replying to the third utterance. Although BART captured  gparted , it failed to handle the action  install . In the second case, we can see that the responses generated by GSN, BERT and BART are general and useless while HeterMPC BERT and HeterMPC BART can still generate a suitable response. Due to the complicated interactions between utterances and interlocutors, the conversation flow might be led by some unnecessary information, which shows the importance of making models aware of the conversation structure. Robustness. The addressee labels are important for constructing a graph used in HeterMPC. This kind of label is commonly available in real life such as A@B labels in group chatting, Twitter, Reddit and various forums that denote speaker A talking to addressee B. However, addressee labels of a part of utterances are missing in the existing MPC datasets since a speaker may forget to specify an addressee. HeterMPC is robust since utterances without addressee labels can be assigned with a general addressee label  To all interlocutors . We leave evaluation on other datasets in future work. We present HeterMPC to model complicated interactions between utterances and interlocutors in MPCs with a heterogeneous graph. Two types of graph nodes and six types of edges are designed. Node-edge-type-dependent parameters are introduced for better utilizing the structural knowledge of conversations during node updating. Results show that HeterMPC outperforms baselines by significant margins, achieving a new state-of-the-art performance for response generation in MPCs on the Ubuntu IRC benchmark. In the future, we will explore better ways of maximizing feature distribution differences to model heterogeneity. We thank anonymous reviewers for their valuable comments.
abstain abstain method method abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain objective abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result abstain other abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result other abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain method abstain abstain abstain objective other