Tangled multi-party dialogue contexts lead to challenges for dialogue reading comprehension, where multiple dialogue threads flow simultaneously within a common dialogue record, increasing difficulties in understanding the dialogue history for both human and machine. Previous studies mainly focus on utterance encoding methods with carefully designed features but pay inadequate attention to characteristic features of the structure of dialogues. We specially take structure factors into account and design a novel model for dialogue disentangling. Based on the fact that dialogues are constructed on successive participation and interactions between speakers, we model structural information of dialogues in two aspects: 1)speaker property that indicates whom a message is from, and 2) reference dependency that shows whom a message may refer to. The proposed method achieves new state-of-the-art on the Ubuntu IRC benchmark dataset and contributes to dialogue-related comprehension. Communication between multiple parties happens anytime and anywhere, especially as the booming social network services hugely facilitate open conversations, such as group chatting and forum discussion, producing various tangled dialogue logs (Lowe et al., 2015; Zhang et al., 2018b; Choi et al., 2018; Reddy et al., 2019; Li et al., 2020a). Whereas, it can be challenging for a new participant to understand the previous chatting log since multi-party dialogues always exhibit disorder and complication (Shen et al., 2006; Elsner and Charniak, 2010; Jiang et al., 2018; Kummerfeld et al., 2019). In fact, it is because of the distributed and random organization, multi-party dialogues are  Corresponding author. I cant ssh into it until I log in as a user. daftykins: oh haha sorry it says connetction refused. Do the ubuntu lts versions able to run as a live cd? regum: Because the root accoutn is disabled in Ubuntu, by default. blackdog: run as live for testing, yes which is really inconvenient, as it means I have to plug a keyboard every time I turn it on. thanks regum: so reconfigure SSHd I meant to the channel and not me, I know nothing about python :) daftykins regum regum blakdog bekks daftykins regum blakdog daftykins jancoow (1003) (1009) (1010) (1011) (1012) (1013) (1014) (1015) (1016)   Figure 1: Here is an example piece of multi-party chatting logs from Ubuntu IRC (Kummerfeld et al., 2019). much less coherent or consistent than plain texts. As the example shown in figure 1, the development of a multi-party dialogue has the following characteristics: 1) Random users successively participate in the dialogue and follow specific topics that they are interested in, motivating the development of those topics. 2) Users reply to former related utterances and mention involved users, forming dependencies among utterances. As a result, multiple ongoing conversation threads grow as the dialogue proceeds, which breaks the consistency and hinders both humans and machines from understanding the context, let alone giving a proper response (Jiang et al., 2018; Kummerfeld et al., 2019; Joty et al., 2019; Jiang et al., 2021). In a word, the behavior of speakers determines the structure of a dialogue passage. And the structure causes problems of reading comprehension. Hence, for better understanding, structural features of dialogue context deserve special attention. messages or clustering conversation threads help with screening concerned parts among contexts, therefore it may be naturally required by passage comprehension, and related downstream dialogue tasks (Elsner and Charniak, 2010; Jia et al., 2020; Liu et al., 2021a), such as response selection, question-answering, etc. Nevertheless, existing works on dialogue disentanglement (Zhu et al., 2020; Yu and Joty, 2020; Li et al., 2020b) generally ignore or pay little attention to characters of dialogues. Earlier works mainly depend on feature engineering (Kummerfeld et al., 2019; Elsner and Charniak, 2010; Yu and Joty, 2020), and use well-constructed handcrafted features to train a naive classifier (Elsner and Charniak, 2010) or linear feed-forward network (Kummerfeld et al., 2019). Recent works are mostly based on two strategies: 1) two-step (Mehri and Carenini, 2017; Zhu et al., 2020; Yu and Joty, 2020; Li et al., 2020b; Liu et al., 2021a) and 2) end-to-end (Tan et al., 2019; Liu et al., 2020a). In terms of the two-step method, the disentanglement task is divided into matching and clustering . It means firstly matching utterance pairs to detect reply-to relations and then clustering utterances according to the matching score. In the end-to-end strategy, alternatively, for each conversation thread, the state of dialogue is modeled, and is mapped with a subsequent utterance to update. At the same time, the subsequent utterance is judged to belong to the best-matched thread. Nonetheless, the essence of both strategies is to model the relations of utterance pairs. Recently, Pre-trained Language Models (PrLMs) (Devlin et al., 2019; an, 2019; Clark et al., 2020) have brought prosperity to numbers of natural language processing tasks by providing contextualized backbones. Various works have reported substantial performance gains with the contextualized information from PrLMs (Lowe et al., 2015; Li et al., 2020a; Liu et al., 2021c; Jia et al., 2020; Wang et al., 2020). Studies on dialogue disentanglement also get benefit from PrLMs (Li et al., 2020b; Zhu et al., 2020), whereas, there is still room for improvement due to their insufficient enhancement of dialogue structure information. So as to enhance characteristic structural features of tangled multi-party dialogues, we design a new model as a better solution for dialogue disentanglement. Structure of a multi-party dialogue is based on the actions of speakers according to the natural development of dialogues. Hence, we model two structural features to help with the detection of reply-to relationships: 1)user identities of messages, referred to as speaker property ; and 2) mention of users in messages, called reference dependency . With the two features enhanced between encoding and prediction, the model makes progress on dialogue disentanglement. Evaluation is conducted on DSTC-8 Ubuntu IRC dataset (Kummerfeld et al., 2019), where our proposed model achieves new state-of-the-art. Further analyses and applications illustrate the advantages and scalability additionally. Our source code is available 1 . Dialogue understanding brings challenges to machine reading comprehension (MRC), in terms of handling the complicated scenarios from multiple speakers and criss-crossed dependencies among utterances (Lowe et al., 2015; Yang and Choi, 2019; Sun et al., 2019; Li et al., 2020a). A dialogue is developed by all involved speakers in a distributed way. An individual speaker focuses on some topics that are discussed in the conversation, and then declares oneself or replies to utterances from related speakers. Therefore, consistency and continuity are broken by tangled reply-to dependencies between non-adjacent utterances (Li et al., 2020a; Jia et al., 2020; Ma et al., 2021; Li et al., 2021), leading to a graph structure that is different from smooth presentation in plain texts. PrLMs have made a significant breakthrough in MRC, where various training objectives and strategies (Devlin et al., 2019; Clark et al., 2020; an, 2019; Lan et al., 2020) have achieved further improvement. Devoted to MRC tasks, PrLMs usually work as a contextualized encoder with some task-oriented decoders added (Devlin et al., 2019). And this paradigm may be a generic but suboptimal solution, especially for some distinctive scenarios, such as dialogue. Recently, numbers of works of dialogue-related MRC have managed to enhance dialogue structural features in order to deal with dialogue passages better (Liu et al., 2021c; Jia et al., 2020; Zhang and Zhao, 2021; Ma et al., 2021; Li et al., 2021), 1 https://github.com/xbmxb/ StructureCharacterization4DD 286 which achieve progress compared to methods that were previously proposed for plain texts. This inspiration impacts and promotes a wide range of dialogue-related MRC tasks such as response selection (Gu et al., 2020; Liu et al., 2021c), question answering (Ma et al., 2021; Li et al., 2021), emotion detection (Hu et al., 2021), etc. 2.2 Dialogue Disentanglement Dialogue disentanglement (Elsner and Charniak, 2010), which is also referred to as conversation management (Traum et al., 2004) , thread detection (Shen et al., 2006) or thread extraction (Adams, 2008), has been studied for decades, since understanding long multi-party dialogues remains to be non-trivial. Thus, dialogue disentanglement methods have been proposed to cluster utterances. Early works can be summarized as feature encoder and clustering algorithms. Well-designed handcraft features are constructed as input of simple networks that predict whether a pair of utterances are alike or different, and clustering methods are then borrowed for partitioning (Elsner and Charniak, 2010; Jiang et al., 2018). Researches are facilitated by a large-scale, high-quality public dataset, Ubuntu IRC, created by Kummerfeld et al. (2019). And then the application of FeedForward network and pointer network (Vinyals et al., 2015) leads to significant progress, but the improvement still partially relies on handcraft-related features (Kummerfeld et al., 2019; Yu and Joty, 2020). Then the end-to-end strategy is proposed and fills the gap between the match and clustering (Liu et al., 2020a), where dialogue disentanglement is modeled as a dialogue state transition process. The utterances are clustered by mapping with the states of each dialogue thread. Inspired by achievements of pre-trained language models (Devlin et al., 2019; Clark et al., 2020; an, 2019), latest work use BERT to contextually encode the dialogue context (Zhu et al., 2020; Li et al., 2020b). Liu et al. (2021b) investigates disentanglement from a different perspective. Their end-to-end co-training approach provides a novel unsupervised baseline. However, attention paid to the characteristics of dialogues seems to be inadequate. Feature engineering-based works represent properties of individual utterances such as time, speakers, and topics with naive handcraft methods, thus ignoring dialogue contexts (Elsner and Charniak, 2010; Kummerfeld et al., 2019). PrLM-based Masked Hierarchical Transformer (Zhu et al., 2020) utilizes the golden conversation structures to operate attentions on related utterances when training models, which results in exposure bias. DialBERT (Li et al., 2020b), a recent architecture including a BERT (Devlin et al., 2019) and an LSTM (Hochreiter and Schmidhuber, 1997), models contextual clues but no dialogue-specific features, and claims a state-of-the-art performance. Our approach draws inspiration from these works and further models structural features for better dialogue understanding. Unlike the above studies, our work incorporates dialogue-specific characters. We propose a new model considering structural characteristics of dialogues, based on the fact that dialogues are developed according to the behavior of speakers. In detail, we model dialogue structures with two highlights: 1) speaker properties of each utterance and 2) reference of speakers between utterances, which both help with modeling inherent interactions among a dialogue passage. Speaker role, as a feature of dialogue passage, has received growing attention recently. On the one hand, speaker embedding facilities research of dialogues. Speaker-aware modeling has also made contributions to response retrieval (Gu et al., 2020; Liu et al., 2021c). SA-BERT (Gu et al., 2020) add a speaker embedding to the input of a PrLM, while MDFN (Liu et al., 2021c) modifies self-attention to enhance speaker switches. Persona has been utilized for smoother dialogue generation. In recent work (Liu et al., 2020b), the speaker-aware information is modeled by adding a reward of persona proximity to the reinforcement learning of generation, based on a persona-annotated dataset (Zhang et al., 2018a). On the other hand, speakers role is a valuable research object for personal knowledge analysis, since the persona can be extracted from one's words in dialogues. Relationship prediction task has been better handled through observing interactions of dialogue speakers (Jia et al., 2021; Tigunova et al., 2021). Tigunova et al. (2021) make use of speaker identity by a SA-BERT (Gu et al., 2020)-like embedding but in utterance-level representation. Relations between utterances have been studied for a long time. Earlier works mostly based on pioneer datasets, Penn Discourse TreeBank 287 (Prasad et al., 2008) and Rhetorical Structure Theory Discourse TreeBank (Mann and Thompson, 1988). In the dialogue field, the much more complex relations contain latent features (Shi and Huang, 2019; Zhang and Zhao, 2021; Jia et al., 2020). Due to the inherent graph structure, Graph Convolutional Network (Kipf and Welling, 2017) is well applied to natural language modeling. Derivations such as Relational-GCN (Schlichtkrull et al., 2018), TextGCN (Yao et al., 2019), LBGCN (Huang et al., 2021), etc, encourage better structural solutions in NLP. In this work, we aim to inject speaker-aware and reference-aware characteristic features for the motivation of disentanglement, instead of making progress on embedding approaches. The definition of the dialogue disentanglement task and details of our model are sequentially presented in this section, illustrating how we make efforts for disentanglement with dialogue structural features. Suppose that we perform disentanglement to a long multi-party dialogue history D = { u 0 , u 2 , . . . , u n } , where D is composed of n utterances. An utterance includes an identity of speaker and a message sent by this user, thus denoted as u i = { s i , m i } . As several threads are flowing simultaneously within D , we define a set of threads T = { t 0 , t 2 , . . . , t p } as a partition of D , where t i = { u i 0 , . . . , u i k } denoting a thread of the conversation. In this task, we aim to disentangle D into T . As indicated before, a multi-party dialogue is constructed by successive participation of speakers, who often reply to former utterances of interest. Thus, a dialogue passage can be modeled as a graph structure whose vertices denote utterances and edges denote reply-to relationships between utterances. Following the two-step method (Mehri and Carenini, 2017), we focus on finding a parent node for each utterance through inference of reply-to relationship, so as to discover edges and then determine the graph of a conversation thread. Figure 2 shows the architecture of the proposed model, which is introduced in detail in this part. The model architecture consists of three modules, including text encoder, structural interaction, and context-aware prediction: 1) The utterances from a dialogue history are encoded with a PrLM, whose output is then aggregated to context-level. 2) The representation is sequentially fed into the structural modeling module, where dialogue structural features are used to characterize contexts. 3) Then in the prediction module, the model performs a fusion and calculates the prediction of reply-to relationships. Pairwise encoding Following previous works (Zhu et al., 2020; Li et al., 2020b), we utilize a pre-trained language model e . g . BERT (Devlin et al., 2019) as an encoder for contextualized representation of tokens. Since chatting records are always long and continuous, it is inappropriate and unrealistic to concatenate the whole context as input. Hence, we focus on the pair of utterances with a reply-to relation. An utterance is concatenated with each parent candidate as input to a PrLM. This may sacrifice contextual information between candidates, but we make up for this in 3.2.3. Assuming that for an utterance u i , we consider former C utterances (including u i itself) as candidates for parent node of u i , the input of a PrLM is in the form of [CLS] u i  j [SEP] U i [SEP] , where 0  j  C  1 . The output is denoted as H 0  RC  L  D , where C denotes the window length in which former utterances are considered as candidates of the parent, L denotes the input sequence length in tokens, D denotes the dimension of hidden states of the PrLM. Note that there is a situation where the golden parent utterance is beyond the range of [ u i  ( C  1) , u i ] . We label a self-loop for u i in this case, which means being too far from the parent making u i a beginning of a new dialogue thread. It makes sense in the real world, because when users join in a chat ( e . g . entering a chatting room), they intend to check a limited number of recent messages and make replies, instead of scanning the entire chatting record. Utterance Aggregation H 0 is pairwise contextualized representations of each pair of token sequences ( u i  j , u i ) , thus need to be aggregated to context-level representation for further modeling. Since special token [CLS] makes more sense on classification tasks (Devlin et al., 2019), we simply reserve the representations of [CLS] . The 288 Encoder (1009) (1010) (1011) (1012) (1013) (1014) (1015) (1016) 1009100910101010101110111012101210131013101410141015101510161016 10091009101010101011101110121012 10131013 101410141015101510161016 10091009 10101010 10111011 10141014 10151015 10161016 (1003)    10171017 10171017 10031003 10031003  10031003 10131013 10121012 10171017 Speaker Classifier (1017) Dependency Aggregation PrLM regum: which is really inconvenient, as it means I have to plug a keyboard every time I turn it on. concatenated pairwise context-level representations from all candidates is denoted as H 1  RC  D , where C denotes the window length and D denotes the dimension of hidden states of the PrLM. For our structural modeling, a simple but effective method is preferred. Hence, for speaker property, we applied the idea of masked MHSA method (Liu et al., 2021c) for better effectiveness and conciseness (Ma et al., 2021). In dependency modeling, we only built one relation type, i.e., reference, where a vanilla r-GCN (Schlichtkrull et al., 2018) is an appropriate baseline method. Speaker Property Modeling We use the term Speaker Property to denote the user identity from whom an utterance is, in formulation, s i . Modeling speaker property could be worthwhile because sometimes a participant may focus on conversations with specific speakers. Following the idea of masking attention (Liu et al., 2021c), we build a Multi-Head Self-Attention (MHSA) mechanism to emphasize correlations between utterances from the same speaker. The mask-based MHSA is formulated as follows: A ( Q, K, V, M ) = softmax( QKT  d k + M ) V , head t = A ( HW Qt , HW Kt , HW Vt , M ), MHSA ( H, M ) = [ head 1 , , . . . ,head N ] WO , (1) where A , head t , Q , K , V , M , N denote the attention, head, query, key, value, mask, and the number of heads, respectively. H denotes the input matrix, and W Qt , W Kt , W Vt , WO are parameters. Operator [  ,  ] denotes concatenation. At this stage, the input of MHSA is the aggregated representation H 1 with a speaker-aware mask matrix M . The element at the i -th row, j -th column of M depend on speaker properties of u i and u j : M[i, j] = (cid:40) 0 , s i = s j  , otherwise H 2 = MHSA ( H 1 , M ), (2) The output of MHSA, HMHSA , has the same dimension with H 1  RC  D . We concatenate H 1 and HMHSA and adjust to the same size using a linear layer, resulting in an output of this module denoted as H 2  RC  D . Reference Dependency Modeling As discussed above, the relation of references between speakers is the most important and straightforward dependency among utterances. Because references indicate interactions between users, it is the internal motivation of the development of a dialogue. To this end, we build a matrix to label the references, which is regarded as an adjacency matrix of a graph representation. In the graph of references, a vertice denotes an utterance and an edge for reference dependence. For example, u 1012 in Figure 1 mentions and reply to regum , forming dependence to utterances from regum , i.e., u 1009 , u 1010 , and u 1014 . Thus there are edges from v 1012 to v 1009 , v 1010 , and v 1014 . Impressed by the significant influence of graph convolutional network (GCN) (Kipf and Welling, 2017), we borrow the relation-modeling of relational graph convolutional network (r-GCN) (Schlichtkrull et al., 2018; Shi and Huang, 2019) in order to enhance the reference dependencies, which can be denoted 289 Model VI ARI 1-1 F1 P R Test Set FeedForward (Kummerfeld et al., 2019) 91.3  75.6 36.2 34.6 38.0  10 union (Kummerfeld et al., 2019) 86.2  62.5 33.4 40.4 28.5  10 vote (Kummerfeld et al., 2019) 91.5  76.0 38.0 36.3 39.7  10 intersect (Kummerfeld et al., 2019) 69.3  26.6 32.1 67.0 21.1 Elsner (Elsner and Charniak, 2008) 82.1  51.4 15.5 12.1 21.5 Lowe (Lowe et al., 2017) 80.6  53.7 8.9 10.8 7.6 BERT (Li et al., 2020b) 90.8 62.9 75.0 32.5 29.3 36.6 DialBERT (Li et al., 2020b) 92.6 69.6 78.5 44.1 42.3 46.2 +cov (Li et al., 2020b) 93.2 72.8 79.7 44.8 42.1 47.9 +feature (Li et al., 2020b) 92.4 66.6 77.6 42.2 38.8 46.3 +future context (Li et al., 2020b) 92.3 66.3 79.1 42.6 40.0 45.6 Ptr-Net (Yu and Joty, 2020) 92.3 70.2  36.0 33.0 38.9 + Joint train (Yu and Joty, 2020) 93.1 71.3  39.7 37.2 42.5 + Self-link (Yu and Joty, 2020) 93.0 74.3  41.5 42.2 44.9 + Joint train&Self-link (Yu and Joty, 2020) 94.2 80.1 -44.5 44.9 44.2 BERT base (Our baseline) 91.4 60.8 74.4 37.2 34.0 41.2 Our model 94.6 +3.2 76.8 +16 84.2 +9.8 51.7 +14.5 51.8 +17.8 51.7 +10.5 Dev Set Decom. ri where B is the set of relationships, which in our module is only reference dependencies. N ri denotes the set of neighbours of vertice v i , which are connected to v i through relationship r , and c i,r is constant for normalization. W ( l ) r and W ( l ) 0 are parameter matrix of layer l . is activated function, which in our implementation is ReLU (Glorot et al., 2011; Agarap, 2018). H 2 is fed into this module and derives H 3  RC  D through r-GCN. The structure-aware representation H 3 needs to be combined with the original representation of [CLS] H 0 for enhancement. An LSTM-like layer (Hochreiter and Schmidhuber, 1997; Li et al., 2020b) can be utilized for compensating contextualized information of the whole candidate window. Motivated by the two points above, we employ a Syn-LSTM module (Xu et al., 2021), which was originally proposed for named entity recognition (NER). A Syn-LSTM is distinguished from an additional input gate for an extra input source, whose parameters are trainable, achieving a better fusion of two input sources. Thus, a layer of Syn-LSTM models the contextual information while the reference dependency is highlighted, enriching relations among parent candidates. In a Syn-LSTM cell, the cell state is derived from the two input and former state as well: c 1 t = tanh(W ( k ) x 1 t + U ( k ) h t  1 + b k ), c 2 t = tanh(W ( p ) x 2 t + U ( p ) h t  1 + b p ), c t = f t  c t  1 + i 1 t  c 1 t + i 2 t  c 2 t , h t = o t  tanh(c t ), where f t , o t , i 1 t , i 2 t are forget gate, output gate and two input gates. c t  1 , c t denote former and current cell states. h t  1 is former hidden state. And W, U, b are learnable parameters. We use the Syn-LSTM in a bi-directional way, and the output is denoted as H 4  RC  2 D r , where D r is the hidden size of the Syn-LSTM. At this stage, H 4 is the structural feature-enchanced representation of each pair of the utterance U i and a candidate parent utterance u i  j . To measure the correlations of these pairs, we follow previous work (Li et al., 2020b) to consider 290 the Siamese architecture between each [ u i , u i  j ] pair ( 1  j  C  1 ) and [ u i , u i ] pair: H 5 [ j ] = [p ii , p ij , p ii  p ij , p ii  p ij ] , where p ij is the representation for the pair of [ U i , U i  j ] from H 4 , and we got H 4  RC  8 D r . H 5 is then fed into a classifier to predict the most correlated pair and predict the parent. Cross-entropy loss is used as the model training objective. Our proposed model is evaluated on a large-scale multi-party dialogue log dataset Ubuntu IRC (Kummerfeld et al., 2019), which is also used as a dataset of DSTC-8 Track2 Task4. The results show that our model surpasses the baseline significantly and achieves a new state-of-the-art. Ubuntu IRC (Internet Relay Chat) (Kummerfeld et al., 2019) is the first available dataset and also the largest and most influential benchmark corpus for dialogue disentanglement, which promotes related research heavily. It is collected from #Ubuntu and #Linux IRC channels in the form of chatting logs. The usernames of dialogue participants are reserved, and reply-to relations are manually annotated in the form of (parent utterance, son utterance) . Table 2 shows statics of Ubuntu IRC. Reply-to relations We calculate the accuracy for the prediction of parent utterance, indicating the inference ability for reply-to relations. Disentanglement For the goal of dialogue disentanglement, threads of a conversation are formed by clustering all related utterances bridged by reply-to relations, in other words, a connected subgraph. At this stage, we use metrics to evaluate following DSTC-8, which are scaled-Variation of Information (VI) (Kummerfeld et al., 2019), Adjusted rand index (ARI) (Hubert and Arabie, Model VI ARI 1-1 F1 P R BERT base 91.7 74.6 80.2 33.5 32.16 35.0 Ablation study + speaker 94.0 81.2 84.9 45.0 44.7 45.3 + reference 94.1 82.4 85.6 47.4 47.4 47.4 + Both 94.4 81.8 86.1 52.6 51.0 54.3 Aggregation methods w/ max-pooling 94.1 80.0 85.3 50.8 52.5 49.2 w/ [CLS] 94.4 81.8 86.1 52.6 51.0 54.3 Layers of Syn-LSTM w/ 1 layer 94.4 81.8 86.1 52.6 51.0 54.3 w/ 2 layers 94.0 78.2 84.6 50.4 50.9 50.0 w/ 3 layers 94.3 79.6 85.3 52.2 51.9 52.6 Table 3: Results of architecture optimizing experiments. 1985), One-to-One Overlap (1-1) (Elsner and Charniak, 2010), precision (P), recall (R), and F1 score of clustering. Note that in the table of results, we present 1-VI instead of VI (Kummerfeld et al., 2019), thus for all metrics, we expect larger numerical values that mean stronger performance. Our implementations are based on Pytorch and Transformers Library (Wolf et al., 2020). We fine-tune the model employing AdamW (Loshchilov and Hutter, 2019) as the optimizer. The learning rate begins with 4e-6. In addition, due to the tradeoff for computing resources, the input sequence length is set to 128, which our inputs are truncated or padded to, and the window width of considered candidates is set to 50. As is presented in Table 1, the experimental results show that our model outperforms all baselines by a large margin as the annotated difference values. It is also shown that our model achieves superior performance on most metrics compared to previously proposed models as highlighted in the table, making a new state-of-the-art (SOTA). We study the effect of speaker property and reference dependency respectively to verify their specific contribution. We ablate either of the characters and train the model. Results in Table 3 show that both speaker property and reference dependency are non-trivial. At the stage of aggregation heading for context-level representations, we consider the influence of different methods of aggregation, i.e., max-pooling and extraction of [CLS] tokens, the models are trained with the same hyper-parameters. Results in Table 3 show [CLS] tokens is a better representation. To determine the optimal depth of the Bi-Syn-LSTM, we do experiments on the number of layers of a Syn-LSTM, also with the same hyper-parameters. According to the results, as shown in Table 3, we put a one-layer Bi-Syn-LSTM for better performance. To intuitively show and discuss the advantages of the proposed approach, we analyze predictions made by our model and the baseline model (i.e., BERT) in the following aspects. 1) We categorize reply-to relationships based on the length of their golden spans (in utterances), and compute the precision of the baseline model and ours. Figure 3a shows that our model outperforms baseline by larger margins on links with longer spans (longer than 20 utterances), indicating that our model is more robust on the longer passages. 2) We select bad cases of the baseline model to find out how the structure-aware modeling benefits dialogue disentanglement. We study predictions from our model on these bad cases. As depicted in Figure 3b, the model well solves 43.3% bad cases. Our model is observed to correct 20.8% bad cases whose utterance pairs are from the same speakers, and 18.3% bad cases whose utterance pairs have a reference. As the illustration shows, our model effectively captures the structural features caused by speaker property and reference dependency, thus gaining improvement. 56.7% predictions are still wrong. It may suggest deeper inner relationships remain to be studied. The used metrics are explained and analyzed briefly for a better understanding of model performance in Appendix A.1. Empirically, it is consistent with our intuition that clarifying the structure of a passage helps with reading comprehension. This section studies the potential of dialogue disentanglement by conducting experiments on different tasks and domains. The dataset of DSTC7 subtask1 (Gunasekara et al., 2019) is a benchmark of response selection tasks, derived from Ubuntu chatting logs, which is challenging because of its massive scale. As shown in Table 4, it contains hundreds of thousand dialogue passages, and each dialogue has speaker-annotated messages and 100 response candidates. In the implementation, pre-processed context passages are firstly fed into the trained model for disentanglement to obtain predicted partitions of context utterances. Then when dealing with the response selection task, we add a self-attention layer to draw attention between utterances within a common cluster in the hope of labels of clusters leading to better contributions to performance. We also make efforts to apply disentanglement on span extraction tasks of question answering datasets, where we consider multi-party dialogue dataset Molweni (Li et al., 2020a), a set of speaker-annotated dialogues with some questions whose answers can be extracted from contexts, which is also collected from Ubuntu chatting logs 4. Because passages in Molweni are brief compared to other datasets we used, utterances tend to belong to the same conversation session through crisscrossed relations. Thus we alternatively leverage labels of reply-to relations from our model, and build graphs among utterances. As the former two datasets are both extracted Ubuntu IRC chatting logs, we additionally consider an open-domain dataset, FriendsQA (Yang and Choi, 2019). It contains daily spoken languages from the TV show Friends 4. FriendsQA gives QA questions and is handled in the same way as the Molweni dataset. Results of the above experiments are presented in Table 5. It is shown that the disentanglement model brings consistent profits to downstream tasks. Yet, gains on FriendsQA are less impressive, indicating domain limitations to some extent. Here we only consider naive baselines and straightforward methods for simplicity and fair comparison, which suggests there is still latent room for performance improvement in future work. In this paper, we study disentanglement on long multi-party dialogue records and propose a new model by paying close attention to the characteristics of dialogue structure, i.e., the speaker property and reference dependency. Our model is evaluated on the largest and latest benchmark dataset Ubuntu IRC, where experimental results show a new SOTA performance and advancement compared to previous work. In addition, we analyze the contribution of each structure-related feature by ablation study and the effect of the different model architecture. Our work discloses that speaker and dependency-aware structural characters are significant and deserve studies in multi-turn dialogue modeling.
abstain abstain objective result abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain abstain abstain abstain abstain abstain other other other other other other other other other other other other other other other other other other other other other other other abstain abstain objective abstain other other other other other other other other other other other other other other objective method method abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain other abstain abstain method method method abstain abstain other abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain method abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result method method abstain method result abstain abstain method abstain method abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method method method abstain abstain abstain abstain abstain method objective objective objective result