Although neural models have achieved competitive results in dialogue systems, they have shown limited ability in representing core semantics, such as ignoring important entities. To this end, we exploit Abstract Meaning Representation (AMR) to help dialogue modeling. Compared with the textual input, AMR explicitly provides core semantic knowledge and reduces data sparsity. We develop an algorithm to construct dialogue-level AMR graphs from sentence-level AMRs and explore two ways to incorporate AMRs into dialogue systems. Experimental results on both dialogue understanding and response generation tasks show the superiority of our model. To our knowledge, we are the first to leverage a formal semantic representation into neural dialogue modeling. Dialogue systems have received increasing research attention (Wen et al., 2015; Serban et al., 2017; Bao et al., 2020), with much recent work focusing on social chats (Ritter et al., 2011; Li et al., 2017) and task-oriented dialogues (Wen et al., 2017; Dinan et al., 2019). There are two salient subtasks in dialogue modeling, namely dialogue understanding (Choi et al., 2018; Reddy et al., 2019; Yu et al., 2020) and response generation (Li et al., 2017; Budzianowski et al., 2018). The former refers to understanding of semantic and discourse details in a dialogue history, and the latter concerns making a fluent, novel and coherent utterance. The current state-of-the-art methods employ neural networks and end-to-end training (Sutskever et al., 2014; Bahdanau et al., 2015) for dialogue modeling. For instance, sequence-to-sequence models have been used to encode a dialogue history, before directly synthesizing the next utterance (Vinyals and Le, 2015; Wen et al., 2017; Bao et al., Dialogue History: SPEAKER-1 : Recently, I've been obsessed with horror films. SPEAKER-2 : Oh, how can you be infatuated with horror films? They're so scary . SPEAKER-1 : Yeah, you are right I used to not watch horror films, but after seeing Silence of the Lamb with Mike last month, I fell in love with them. SPEAKER-2 : It's amazing. But if I were you, I wouldn't have the courage to watch the first one. SPEAKER-1 : But it's really exciting . Ground-Truth : Maybe, but I would rather watch romance, science fiction, crime or even disaster movie instead of a horror picture Transformer : Great. I'm looking forward to it. I just can't keep away from the food that I saw. Figure 1: A conversation from DailyDialog. Some important contents are marked with squares. 2020). Despite giving strong empirical results, neural models can suffer from spurious feature associations in their neural semantic representation (Poliak et al., 2018; Kaushik et al., 2020), which can lead to weak robustness, inducing irrelevant dialogue states (Xu and Sarikaya, 2014; Sharma et al., 2019; Rastogi et al., 2019) and generating unfaithful or irrelevant text (Maynez et al., 2020; Niu and Bansal, 2020). As shown in Figure 1, the baseline Transformer model pays attention to the word  lamb  but ignores its surrounding context, which has important contents (marked with squares) that indicate its true meaning, thereby giving an irrelevant response that is related to food. Intuitively, such issues can be alleviated by having a structural representation of semantic information, which treats entities as nodes and builds structural relations between nodes, making it easy to find the most salient context. Explicit structures are also more interpretable compared to neural representation and have been shown useful for information extraction (Strubell et al., 2018; Sun et al., 2019; Li et al., 2020; Bai et al., 2021; Sachan et al., 2021), summarization (Liu et al., 2015; Hardy and Vlachos, 2018; Liao et al., 2018) and machine translation (Marcheggiani et al., 2018; Song et al., 2019a). We explore AMR (Banarescu et al., 2013) as a semantic representation for dialogue histories in order to better represent conversations. As shown in the central block of Figure 2, AMR is one type of sentential semantic representations, which models a sentence using a rooted directed acyclic graph, highlighting its main concepts ( e.g.  mistake ) and semantic relations ( e.g. ,  ARG0  1 ), while abstracting away function words. It can thus potentially offer core concepts and explicit structures needed for aggregating the main content in dialogue. In addition, AMR can also be useful for reducing the negative influence of variances in surface forms with the same meaning, which adds to data sparsity. Existing work on AMR parsing focuses on the sentence level. However, as the left block of Figure 2 shows, the semantic structure of a dialogue history can consist of rich cross-utterance coreference links (marked with squares) and multiple speaker interactions. To this end, we propose an algorithm to automatically derive dialogue-level AMRs from utterance-level AMRs, by adding cross-utterance links that indicate speakers, identical mentions and co-reference links. One example is shown in the right block of Figure 2, where newly added edges are in color. We consider two main approaches of making use of such dialogue-level AMR structures. For the first method, we merge an AMR with tokens in its corresponding sentence via AMR-to-text alignments, before encoding the resulting structure using a graph Transformer (Zhu et al., 2019). For the second method, we separately encode an AMR and its corresponding sentence, before leveraging both representations via feature fusion (Mangai et al., 2010) or dual attention (Cal-ixto et al., 2017). We verify the effectiveness of the proposed framework on a dialogue relation extraction task (Yu et al., 2020) and a response generation task (Li et al., 2017). Experimental results show that the proposed framework outperforms previous 1 Please refer to PropBank (Kingsbury and Palmer, 2002; Palmer et al., 2005) for more details. methods (Vaswani et al., 2017; Bao et al., 2020; Yu et al., 2020), achieving the new state-of-the-art results on both benchmarks. Deep analysis and human evaluation suggest that semantic information introduced by AMR can help our model to better understand long dialogues and improve the coherence of dialogue generation. One more advantage is that AMR is helpful to enhance the robustness and has a potential to improve the interpretability of neural models. To our knowledge, this is the first attempt to leverage the AMR semantic representation into neural networks for dialogue understanding and generation. Our code is available at https://github.com/muyeby/AMR-Dialogue . Figure 2 illustrates our method for constructing a dialogue-level AMR graph from multiple utterance-level AMRs. Given a dialogue consisting multiple utterances, we adopt a pretrained AMR parser (Cai and Lam, 2020) to obtain an AMR graph for each utterance. For utterances containing multiple sentences, we parse them into multiple AMR graphs, and mark them belonging to the same utterance. We construct each dialogue AMR graph by making connections between utterance AMRs. In particular, we take three strategies according to speaker, identical concept and co-reference information. Speaker We add a dummy node and connect it to all root nodes of utterance AMRs. We add speaker tags ( e.g. , SPEAKER 1 and SPEAKER 2) to the edges to distinguish different speakers. The dummy node ensures that all utterance AMRs are connected so that information can be exchanged during graph encoding. Besides, it serves as the global root node to represent the whole dialogue. Identical Concept There can be identical mentions in different utterances ( e.g.  possible  in the first and the forth utterances in Figure 2), resulting in repeated concept nodes in utterance AMRs. We connect nodes corresponding to the same non-pronoun concepts by edges labeled with SAME 2 . This type of connection can further enhance cross-sentence information exchange. 2 Compared with co-reference, identical concept relations can connect different words which share the same meaning e.g. (cid:104) could , might (cid:105) , (cid:104) fear , afraid (cid:105) . which are frequent in conversations (Grosz et al., 1995; Newman et al., 2008; Quan et al., 2019). We conduct co-reference resolution on dialogue text using an off-to-shelf model 3 in order to identify concept nodes in utterance AMRs that refer to the same entity. For example, in Figure 2,  I  in the first utterance, and  sir  in the second utterance refer to the same entity, SPEAKR 1. We add edges labeled with COREF between them, starting from later nodes to earlier nodes ( later and earlier here refer to the temporal order of ongoing conversation), to indicate their relation 4 . We adopt a standard Transformer (Vaswani et al., 2017) for dialogue history encoding. Typically, a Transformer encoder consists of L layers, taking a sequence of tokens (i.e., dialogue history) S = { w 1 , w 2 , ..., w N } , where w i is the i -th token and N is the sequence length, as input and produces vectorized word representations { h l 1 , h l 2 , ..., h lN } iteratively, l  [1 , ..., L ] . Overall, a Transformer encoder can be written as: H = SeqEncoder ( emb ( S )) , (1) where H = { h L 1 , h L 2 , ..., h Ln } , and emb denotes a function that maps a sequence of tokens into the corresponding embeddings. Each Transformer layer consists of two sub-layers: a self-attention sub-layer and a position-wise feed forward network. The former calculates a set of attention scores:  ij = Attn ( h i , h j ) . which are used to update the hidden state of w i : h li = (cid:88) N j =1  ij ( WV h l  1 j ) , (3) where WV is a parameter matrix. The position-wise feed-forward ( FFN ) layer consists of two linear transformations: FFN ( h ) = W 2 ReLU ( W 1 h + b 1 ) + b 2 , (4) where W 1 , W 2 , b 1 , b 2 are model parameters. We take the dialogue relation extraction task (Yu et al., 2020) as an example. Given a dialogue history S and an argument (or entity) pair ( a 1 , a 2 ), the goal is to predict the corresponding relation type r  R between a 1 and a 2 . We follow a previous dialogue relation extraction model (Chen et al., 2020) to feed the hidden states of a 1 and a 2 (denoted as h a 1 , h a 2 ) into a classifier to obtain the probability of each relation types: P rel = softmax ( W 3 [ h a 1 ; h a 2 ] + b 3 ) , (5) where W 3 and b 3 are model parameters. where  denotes the set of model parameters. In practice, we use BERT (Devlin et al., 2019) for calculating h a 1 and h a 2 , which can be regarded as pre-trained initialization of the Transformer encoder. Given a dialogue history S , we use a standard autoregressive Transformer decoder (Vaswani et al., 2017) to generate a response Y = { y 1 , y 2 , ..., y |Y| } . At time step t , the previous output word y t  1 is firstly transformed into a hidden state s t by a self-attention layer as Equations 2 and 3. Then an encoder-decoder attention mechanism is applied to obtain a context vector from encoder output hidden states { h L 1 , h L 2 , . . . , h LN } :   ti = Attn ( s t , h Li ) , c t = (cid:88) N i =1   ti h Li , (7) The obtained context vector c t is then used to calculate the output probability distribution for the next word y t over the target vocabulary 5 : P voc = softmax ( W 4 c t + b 4 ) , (8) where W 4 , b 4 are trainable model parameters. The k -th value of P voc is the conditional probability of k -th word in vocabulary given a dialogue. Given a dialogue history-response pair {S , Y} , the model minimizes a cross-entropy loss: (cid:96) =  | Y | (cid:88) t =1 logP voc ( y t | y t  1 , ..., y 1 , S ;  ) , (9) where  denotes all model parameters. Our model takes a dialogue history S and the corresponding dialogue AMR as input. Formally, 5 Similar to the encoder, there is also multi-head attention, a position-wise feed-forward layer and residual connections, which we omit in the equations. an AMR is a directed acyclic graph G = (cid:104)V , E(cid:105) , where V denotes a set of nodes (i.e. AMR concepts) and E (i.e. AMR relations) denotes a set of labeled edges. An edge can be further represented by a triple (cid:104) n i , r ij , n j (cid:105) , meaning that the edge is from node n i to n j with label r ij . We consider two main ways of making use of dialogue-level AMRs. The first method (Fig-ure 3(a)) uses AMR semantic relations to enrich a textual representation of the dialogue history. We project AMR nodes onto the corresponding tokens, extending Transformer by encoding semantic relations between words. For the second approach, we separately encode an AMR and its sentence, and use either feature fusion (Figure 3(b)) or dual attention (Figure 3(c)) to incorporate their embeddings. We adopt a Graph Transformer (Zhu et al., 2019) to encode an AMR graph, which extends the standard Transformer (Vaswani et al., 2017) for modeling structural input. A L -layer graph Transformer takes a set of node embeddings { n 1 , n 2 , ..., n M } and a set of edge embeddings { r ij | i  [1 , ..., M ] , j  [1 , ..., M ] } as input 6 and produces more abstract node features { h l 1 , h l 2 , ..., h lM } iteratively, where l  [1 , ..., L ] . The key difference between a graph Transformer and a standard Transformer is the graph attention layer. Compared with self-attention layer (Equation 2), the graph attention layer explicitly considers graph edges when updating node hidden states. For example, give an edge (cid:104) n i , r ij , n j (cid:105) , the attention score   ij is calculated 6 If there is no relation between n i and n j , r ij = None  as:   ij = exp( e ij ) (cid:80) Mm =1 exp ( e im ) ,  e ij = ( WQ h l  1 i ) T ( WK h l  1 j + WR r ij )  d , (10) where WR is a transformation matrix, r ij is the embedding of relation r ij , d is hidden state size, and { h 01 , h 02 , ..., h 0 M } = { n 1 , n 2 , ..., n M } . The hidden state of n i is then updated as: h li = (cid:88) M j =1  ij ( WV h l  1 j + WR r ij ) , (11) where WV is a parameter matrix. We first use the JAMR aligner (Flanigan et al., 2014) to obtain a node-to-word alignment, then adopt the alignment to project the AMR edges onto text with following rules: where A is a one-toK alignment ( K  [0 , . . . , N ] ). In this way, we obtain a projected graph G (cid:48) = (cid:104)V (cid:48) , E (cid:48) (cid:105) , where V (cid:48) represents the set of input words { w 1 , w 2 , ..., w N } and E (cid:48) denotes a set of word-to-word semantic relations. Inspired by previous work on AMR graph modeling (Guo et al., 2019; Song et al., 2019b; Sun et al., 2019), we adopt a hierarchical encoder that stacks a sequence encoder and a graph encoder. A sequence encoder ( SeqEncoder ) transforms a dialogue history into a set of hidden states: HS = SeqEncoder ( emb ( S )) . H  S = GraphEncoder ( HS , emb ( E (cid:48) )) , (15) word representations before and after refinement (as shown in Figure 3(b)): HF = LayerNorm ( HS + H  S ) . where LayerNorm denotes the layer normalization (Ba et al., 2016). We name the hierarchical encoder as Hier , which can be used for both dialogue understanding and dialogue response generation. We consider integrating both text cues and AMR structure cues for dialogue understanding and response generation, using a dual-encoder network. First, a sequence encoder is used to transform a dialogue history S into a text memory (denoted as HS = { h S 1 , h S 2 , ..., h SN } ) using Equation 1. Second, the AMR graph G is encoded into graph memory (denoted as HG = { h G 1 , h G 2 , ..., h GM } ) by a graph Transformer encoder using Equation 12. For dialogue understanding (Figure 3(b)) and dialogue response generation (Figure 3(c)), slightly different methods of feature integration are used due to their different nature of outputs. Dialogue Understanding . Similar to Section 4.2, we first use the JAMR aligner to obtain a node-to-word alignment A . Then we fuse the word and AMR node representations as follows:  h i = (cid:40) f ( h Si , h Gj ) , if  j, A ( n j ) = w i , f ( h Si , h  ) , otherwise , (17) where h  is the vector representation of the dummy node (see Figure 2), f is defined as: h = LayerNorm ( h 1 + h 2 ) . classifier for relation prediction (Equation 5). Dialogue Response Generation . We replace the standard encoder-decoder attention (Equation 7) with a dual-attention mechanism (Song et al., 2019a). In particular, given a decoder hidden state s t at time step t , the dual-attention mechanism calculates a graph context vector c St and a text context vector c Gt , simultaneously:   ti = Attn ( s t , h Si ) ,   tj = Attn ( s t , h Gj ) , c St = (cid:88) N i =1   ti h Si , c Gt = (cid:88) M j =1   tj h Gj , (19) Model data-v1 data-v2 dev test dev test F1 (  ) F 1 c (  ) F1 (  ) F1 c (  ) F1 (  ) F 1 c (  ) F1 (  ) F1 c (  ) AGGCN  46.6(-) 40.5(-) 46.2(-) 39.5 (-) --LSR  44.5(-) -44.4(-) --DHGAT  57.7(-) 52.7(-) 56.1(-) 50.7(-) --BERT 60.6(1.2) 55.4(0.9) 58.5(2.0) 53.2(1.6) 59.4 (0.7) 54.7(0.8) 57.9(1.0) 53.1(0.7) BERT s 63.0(1.5) 57.3(1.2) 61.2(0.9) 55.4(0.9) 62.2(1.3) 57.0(1.0) 59.5(2.1) 54.2(1.4) BERT c 66.8(0.9) 60.9(1.0) 66.1(1.1) 60.2(0.8) 66.2(0.9) 60.5(1.1) 65.1(0.8) 59.8(1.2) Hier 68.2(0.8) 62.2 (0.7) 67.0(0.9) 61.3(0.6) 68.0(0.6) 62.2(0.4) 66.7(0.3) 61.0(0.4) Dual 68.3 (0.6) 62.2 (0.2) 67.3 (0.4) 61.4 (0.2) 68.2 (0.5) 62.3 (0.4) 67.1 (0.4) 61.1 (0.5) Table 1: Performance on DialogRE, where  denotes the standard deviation computed from 5 runs, and  indicates results reported by Chen et al. (2020). and the final context vector  c t is calculated as: c t = W c [ c St ; c Gt ] + b c , (20) where W c and b c are model parameters. We name the dual-encoder model as Dual . We evaluate our model on DialogRE (Yu et al., 2020), which contains totally 1,788 dialogues, 10,168 relational triples and 36 relation types in total. On average, a dialogue in DialogRE contains 4.5 relational triples and 12.9 turns. We report experimental results on both original (v1) and updated (v2) English version. 7 5.1 Settings We adopt the same input format and hyper-parameter settings as Yu et al. (2020) for the proposed model and baselines. In particular, the input sequence is constructed as [CLS] d [SEP] a 1 [SEP] a 2 [SEP] , where d denotes the dialogue, and a 1 and a 2 are the two associated arguments. In the BERT model of Yu et al. (2020), only the hidden state of the [CLS] token is fed into a classifier for prediction, while our baseline (BERT c ) additionally takes the hidden states of a 1 and a 2 . All hyperparameters are selected by prediction accuracy on validation dataset (See Table 6 for detailed hyperparameters). Metrics Following previous work on DialogRE, we report macro F 1 score on relations in both the standard (F 1 ) and conversational settings (F 1 c ; Yu et al., 2020). F 1 c is computed over the first few turns of a dialogue where two arguments are first mentioned. 7 https://dataset.org/dialogre/ 5.2 Main Results Table 1 shows the results of different systems on DialogRE. We compare the proposed model with two BERT-based approches, BERT and BERT s . Based on BERT, BERT s (Yu et al., 2020) highlights speaker information by replacing speaker arguments with special tokens. For completeness, we also include recent methods, such as AGGCN (Guo et al., 2019), LSR (Nan et al., 2020) and DHGAT (Chen et al., 2020). BERT c and Hier , Dual represent our baseline and the proposed models, respectively. By incorporating speaker information, BERT s gives the best performance among the previous system. Our BERT c baseline outperforms BERT s by a large margin, as BERT c additionally considers argument representations for classification. Hier significantly ( p < 0 . 01 ) 8 outperforms BERT c in all settings, with 1.4 points of improvement in terms of F1 score on average. A similar trend is observed under F1 c . This shows that semantic information in AMR is beneficial to dialogue relation extraction, since AMR highlights core entities and semantic relations between them. Dual obtains slightly better results than Hier , which shows effect of separately encoding a semantic structure. Finally, the standard deviation values of both Dual and Hier are lower than the baselines. This indicates that our approaches are more robust regarding model initialization. We split the dialogues of the DialogRE (v2) devset into five groups by the utterance-based distance between two arguments. As shown in Figure 4, Dual gives better results than BERT c except when 8 We use pair-wised t -test. the argument distance is less than 5. In particular, Dual surpasses BERT c by a large margin when the arguments distance is greater than 20. The comparison indicates that AMR can help a model to better handle long-term dependencies by improving the entity recall. In addition to utterance distance, we also consider word distance and observe a similar trend (as shown in Appendix 7). Figure 5 shows a conversation between a manager and an employee who might have taken a leave. The baseline model incorrectly predicts that the relation between two interlocutors is parent and child. It might be influenced by the last sentence in the conversation, assuming that it is a dialogue between family members. However, the proposed model successful predicts the interlocutors' relation, suggesting it can extract global semantic information in the dialogue from a comprehensive perspective. We conduct experiments on the DailyDialog benchmark (Li et al., 2017), which contains 13,119 daily multi-turn conversations. On average, the number of turns for each dialogue is 7.9, and each utterance has 14.6 tokens. We take Transformer as a baseline. Our hyperparameters are selected by word prediction accuracy on validation dataset. The detailed hyperparameters are given in Appendix (See Table 6). Metric We set the decoding beam size as 5 and adopt BLEU-1/2/3/4 (Papineni et al., 2002) and Distinct-1/2 (Li et al., 2016) as automatic evaluation metrics. The former measures the n-gram overlap between generated response and Dialogue : SPEAKER-1 : A new place for a new Ross. I'm gonna have you and all the guys from work over once it's y'know, furnished. SPEAKER-2 : I must say it's nice to see you back on your feet. SPEAKER-1 : Well I am that. And that whole rage thing is definitely behind me. SPEAKER-2 : I wonder if its time for you to rejoin our team at the museum? SPEAKER-1 : Oh Donald that-that would be great. I am totally ready to come back to work. IWhat? No! Wh-What are you doing?!! GET OFF MY SISTER!!!!!!!!!!!!! Ground-Truth : per:boss(S1, S2) Baseline : per:parent(S1, S2) Ours : per:boss(S1, S2) Figure 5: Case study for dialogue relation extraction. the target response while the latter assesses the generation diversity, which is defined as the number of distinct unior bi-grams divided by the total amount of generated words. In addition, we also conduct human evaluation. Following Bao et al. (2020), we ask annotators who study linguistics to evaluate model outputs from four aspects, which are fluency, coherence, informativeness and overall performance. The scores are in a scale of { 0, 1, 2 } . The higher, the better. Table 2 reports the performances of the previous state-of-the-art methods and proposed models on the DailyDialog testset. For the previous methods, PLATO and PLATO w/o L are both Transformer models pre-trained on large-scale conversational data (8.3 million samples) and finetuned on DailyDialog. For completeness, we also report other systems including Seq2Seq (Vinyals and Le, 2015) and iVAE MI (Fang et al., 2019). Among the previous systems, PLATO and PLATO w/o L report the best performances. Our Transformer baseline is highly competitive in terms of BLEU and Distinct scores. Compared with the Transformer baseline, both Dual and Hier show better numbers regarding BLEU and Distinct, and the gains of both models are significant ( p < 0 . 01 ). This indicates that semantic information in AMR graphs is useful for dialogue response generation. In particular, the gains come from better recall of the important entities and their relations in a dialogue history, which can leads to generating a more detailed response. We conduct human evaluation on randomly selected 50 dialogues and corresponding generated responses of the baseline and our models. As shown in Table 3, the Transformer baseline gives the lowest scores, while Dual sees the highest scores from all aspects. Our main advantage is on the Coherence , meaning that AMRs are effective on recalling important concepts and relations. As the result, it makes it easier for our models to generate coherent replies. Examples are shown in Figure 8 in Appendix. Comparatively, all systems achieve high scores regarding Fluency , suggesting that this aspect is not the current bottleneck for response generation. This section contains analysis concerning the effects of graph features, dialogue length and model robustness. We use Dual model for experiments since it gives slightly better results than Hier . Table 4 shows the results of our best performing models on the two datasets regarding different con-figurations on the dialogue AMR graphs. We report the average F1 score for DialogRE and the BLEU-1/Distinct-1 score for DailyDialog. First, using utterance-level AMR improves the text baseline by 1.2 points and 1.5 points with regard to F1 and Setting DialogRE (v2) DailyDialog Dialog-AMR( Dual ) 68.2 38.2/5.9 -Speaker 67.5 37.7/5.7 -Ident. BLEU-1 scores, respectively. This indicates that the semantic knowledge in formal AMR is helpful for dialogue modeling. Second, our manually added relations (in Section 2) also leads to improvements, ranging from 0.5 to 1.0 in BLEU-1 score. The speaker relation is the most important for dialogue relation extraction, a possible reason is that DialogRE dataset mainly focus on person entities. Also, co-reference relations help the most in dialogue response generation. The identical concept relations give least improvements among three relations. Finally, combining all relations to build a Dialog-AMR graph achieves best performance on both datasets. We group the devset of DialogRE (v2) and DailyDialog into five groups according to the number of utterances in a dialogue. Figure 6 summarizes the performance of the baseline and the proposed model on dialogue understanding (DU) and response generation (RG) tasks. In dialogue understanding, our model gives slightly better F1 scores than the baseline when a dialogue has smaller than 12 utterance. The performance improvement is more significant when modeling a long dialogue. This confirms our motivation that AMR can help to understand long dialogues. In dialogue response generation, our model consistently outperforms the Transformer baseline by a large margin on Model Original Paraphrased Baseline 100 94.50 Ours 100 98.50 Table 5: F1 on original and paraphrased testsets. dialogues of different lengths, still with more improvements on larger dialogues. Overall, these results are consistent with Table 1 and 2, showing that AMR can provide useful semantic information and alleviate the issue of long-range dependency. Recent studies show that neural network-based dialog models lack robustness (Shalyminov and Lee, 2018; Einolghozati et al., 2019). We select 100 instances from the testset of DialogRE (v2) where both baseline and our model gives true prediction, before paraphrasing the source dialogues manually (see appendix B.3 for paraphrasing guidelines.). Results on the paraphrased dataset are given in Table 5. The performance of baseline model drop from 100 to 94.5 on paraphrased dataset. By contrast, the result of our model reaches 98.5, 4 points higher than baseline. This confirms our assumption that AMR can reduce data sparsity, thus improve the robustness of neural models. Semantic Parsing for Dialogue Some previous work builds domain-specified semantic schema for task-oriented dialogues. For example, in the PEGASUS (Zue et al., 1994) system, a sentence is first transformed into a semantic frame and then used for travel planing. Wirsching et al. (2012) use semantic features to help a dialogue system perform certain database operations. Gupta et al. (2018) represent task-oriented conversations as semantic trees where intents and slots are tree nodes. They solve intent classification and slot-filling task via semantic parsing. Cheng et al. (2020) design a rooted semantic graph that integrates domains, verbs, operators and slots in order to perform dialogue state tracking. All these structures are designed for specified task only. In contrast, we investigate a general semantic representation for the modeling of everyday conversations. Constructing AMRs beyond Sentence Level There are a few attempts to construct AMRs beyond the sentence level. Liu et al. (2015) construct document-level AMRs by merging identical concepts of sentence-level AMRs for abstractive summerization, and Liao et al. (2018) further extend this approach to multi-document summer-ization. O'Gorman et al. (2018) manually annotate co-reference information across sentence AMRs. We focus on creating conversation-level AMRs to facilitate information exchange more effectively for dialogue modeling. Bonial et al. (2020) adapt AMRs on dialogues by enriching the standard AMR schema with dialogue acts, tense and aspect, and they construct a dataset consisting of 340 dialogue AMRs. However, they propose theoretical changes in the schema for annotating AMRs, while we explore empirical solutions that leverage existing AMRs of the standard schema on dialogues. AMR Parsing and Encoding Our work is also related to AMR parsing (Flanigan et al., 2014; Konstas et al., 2017a; Lyu and Titov, 2018; Guo and Lu, 2018; Zhang et al., 2019; Cai and Lam, 2020) and AMR encoding (Konstas et al., 2017b; Song et al., 2018; Zhu et al., 2019; Song et al., 2020; Zhao et al., 2020; Bai et al., 2020). The former task makes it possible to use automatically-generated AMRs for downstream applications, while the latter helps to effectively exploit structural information in AMRs. In this work, we investigate AMRs for dialogue representation and combine AMRs with text for dialogue modeling. We investigated the feasibility of using AMRs for dialogue modeling, describing an algorithm to construct dialogue-level AMRs automatically and exploiting two ways to incorporate AMRs into neural dialogue systems. Experiments on two benchmarks show advantages of using AMR semantic representations model on both dialogue understanding and dialogue response generation. Yue Zhang is the corresponding author. We would like to thank the anonymous reviewers for their insightful comments and Jinhao Jiang for his help for data preparation. This work has been supported by Tencent AI Lab Rhino-Bird Focused Research Program. It also receives support from the Westlake University and Bright Dream Joint Institute for Intelligent Robotics, and a research grant from Rxhui Inc.
abstain abstain abstain objective result objective abstain abstain abstain abstain other other abstain abstain other abstain abstain abstain abstain abstain abstain objective abstain method abstain method abstain abstain abstain result abstain objective other method abstain abstain method method method method abstain abstain abstain abstain abstain abstain abstain method abstain method method other abstain abstain abstain abstain abstain method abstain other abstain method method abstain other abstain abstain method method abstain abstain method abstain abstain abstain method abstain abstain abstain other abstain abstain other abstain abstain abstain abstain method abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain other abstain abstain method other abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other other other other other other other objective other other other method other objective method other objective objective abstain other other other other