Argument pair extraction (APE) is a research task for extracting arguments from two passages and identifying potential argument pairs. Prior research work treats this task as a sequence labeling problem and a binary clas-sification problem on two passages that are directly concatenated together, which has a limitation of not fully utilizing the unique characteristics and inherent relations of two different passages. This paper proposes a novel attention-guided multi-layer multi-cross encoding scheme to address the challenges. The new model processes two passages with two individual sequence encoders and updates their representations using each other's representations through attention. In addition, the pair prediction part is formulated as a table-filling problem by updating the representations of two sequences' Cartesian product. Furthermore, an auxiliary attention loss is introduced to guide each argument to align to its paired argument. An extensive set of experiments show that the new model significantly improves the APE performance over several alternatives 1 . Mining argumentation structures within a corpus is a crucial task in argument mining research field (Palau and Moens, 2009). There are usually two main components in learning natural language argument structures: (1) detecting argumentative units, (2) predicting relations between the identified arguments. It has been widely studied by natural language processing (NLP) researchers (Cabrio and Villata, 2018) and applied to domains such as: web debating platforms (Boltuzic and Snajder, 2015; Swanson et al., 2015; Chakrabarty et al., 2019),  Liying Cheng is under the Joint Ph.D. persuasive essays (Stab and Gurevych, 2014; Persing and Ng, 2016), social media (Abbott et al., 2016), etc. Unlike traditional argument extraction tasks that are mainly from monologues, Cheng et al. (2020) propose a new task argument pair extraction (APE) from two passages in a new domain, namely peer review process, focusing on exploiting the interactions between reviewer comments and author rebuttals. As shown in Figure 1, APE task aims to extract the argument pairs from two passages. Specific suggestions, questions or challenges in reviews are considered as review arguments. Response sentences that answer or explain the specific review argument are its paired rebuttal arguments. For example in the pink area, the reviewer points out the lack of literature review in submission (i.e., review sentences 11-12). As a response, the authors argue that they select the literature based on the special focus of their work (i.e., rebuttal sentence 6-7). Similar to the two components in the traditional argumentation structure mining, the APE task can be divided into two subtasks: (1) extracting the review and rebuttal arguments from two passages, (2) predicting if an extracted review argument and a rebuttal argument form an argument pair. The first subtask can be cast as a sequence labeling problem and the second one can be cast as a binary classifi-cation problem. One straightforward approach is to couple the two subtasks in a pipeline. However, such a pipeline approach learns two subtasks independently without sharing ample information. To address this limitation, the pioneering work (Cheng et al., 2020) employs a multi-task learning framework to train two subtasks simultaneously. However, there are several shortcomings in the multi-task model. First, the review passage and its rebuttal passage are concatenated as a single passage to perform the argument extraction subtask with sequence labeling. It is obvious to see from 12 8 Rebuttal Review 1 2 3 4 5 6 7 1 2 3 4 5 6 7 O O O O O O B 9 10 11 I I B I I I B O B I I 1 This relatively novel work proposes to augment current RL models by adding self-supervised tasks encouraging better internal representations. Figure 1 that the review and rebuttal passages have their own styles in terms of structure and wording. Hence, it is not suitable to concatenate them as one long sequence, which is against the fact that they are two unique sequences in essence and hinders the model from well-utilizing their different characteristics. To overcome this limitation, we treat review and rebuttal passages as two individual sequences and design two sequence encoders for them respectively. In each sequence encoder, the sequence representations will be updated by the other's representations through mutual attention. It allows us to better distinguish two passages, and meanwhile, to conveniently exchange information between them through the attention mechanism. Second, the subtask coordination capability of their multi-task framework is weak as two subtasks only coordinate with each other via the shared feature encoders, i.e., the sentence encoder for the sequence of word tokens and the passage encoder for the concatenation of sentences. Thus, the shared information between two subtasks is only learned implicitly. To overcome this limitation, we propose an attention-guided multi-layer multi-cross (MLMC) encoding mechanism. Inspired by the table-filling approach (Miwa and Sasaki, 2014), we form a table that represents features for the Cartesian Product of review and rebuttal sequences by utilizing both of their embeddings, as shown in the right portion of Figure 1. The table representations will be updated with the incorporation of the two sequence representations, and in return, it will also help to update the mutual attention mentioned above. It is named as multi-cross encoder because these three encoding components (i.e., one table and two sequences) interact with each other explicitly and extensively. By stacking multiple encoder layers, the two subtasks can further benefit each other. In addition, we also design an auxiliary attention loss to guide each argument to refer to its paired arguments. This additional loss not only enhances the model performance, but also significantly improves the attention interpretability. To summarize, the contributions of this paper are three-fold. Firstly, we apply the table-filling approach to model the sentence-level correlation between two passages with multiple sentences for the first time. Secondly, on the model side, we propose an MLMC encoder to explicitly learn the useful shared information in the two passages. Furthermore, we introduce an auxiliary attention loss, which is able to further improve the efficacy of the mutual attentions. Thirdly, we evaluate our model on the benchmark dataset (Cheng et al., 2020), and the results show that our model achieves a new state-of-the-art performance on the APE task. Argument mining has wide applications in educational domain, including persuasive essays (Stab and Gurevych, 2017; Eger et al., 2017), scientific articles (Teufel et al., 2009; Guo et al., 2011), writing assistance (Zhang and Litman, 2016), essay scoring (Persing and Ng, 2015; Somasundaran et al., 2016), peer reviews (Hua et al., 2019), etc. Unlike previous works, Cheng et al. (2020) introduce a new task named APE in the domain of peer review and rebuttal, which intends to extract the argument pairs from two passages simultaneously. Table-filling approaches (Miwa and Sasaki, 2014; Gupta et al., 2016; Zhang et al., 2017) have been proposed to work towards the joint task of name entity recognition (NER) and relation extraction (RE). In their work, the diagonal entries of the table show the words' entity types and the off-diagonal entries show the relation types with other words. More recently, there are more research works to propose various table-filling models on different tasks. Wang and Lu (2020) propose to learn two separate encoders (a table encoder and a sequence encoder) by interacting with each other for joint NER and RE task. Wu et al. (2020) propose a grid tagging scheme to address the aspect-oriented fine-grained opinion extraction task. Compared to our model, one major difference is the table shape. In their tables, the row and column represent the same sequence, and thus in square shape. In our model, the table is in a rectangle shape where the row and column represent two different sequences with different lengths. Another clear difference is that each entry in their table is for word-pair relation, whereas each entry in our table captures sentence-pair relation. As we can see from Figure 1, the review/rebuttal sequence consists of a list of sentences. Thus, it requires extra effort to learn comprehensive sentence representations. In this paper, we tackle the APE task, which aims to study the internal structure and relations between two passages, e.g., review and rebuttal passages. For example, as shown in Figure 1, given a pair of review passage s rv = [ s rv, 1 ,    , s rv, 12 ] (in the red box) and rebuttal passage s rb = [ s rb, 1 ,    , s rb, 7 ] (in the orange box), we intend to automatically extract all argument pairs between them. First, for the argument mining subtask, we cast it as a sentence-level sequence labeling problem following the work (Cheng et al., 2020) using the standard BIO scheme (Ramshaw, 1995; Ratinov and Roth, 2009). This subtask segments the argumentative units (highlighted in blue/pink) from nonargumentative units (highlighted in grey) for each passage. The label sequences for the review passage and the rebuttal passage are shown in the right portion of Figure 1. Second, the sentence pairing subtask predicts whether the two sentences belong to one argument pair. Here, we formulate it as a table-filling problem following the work (Miwa and Sasaki, 2014). Take the 8 th review sentence s rv, 8 in the first review argument as an example, the rebuttal argument sentences { s rb, 2 , s rb, 3 , s rb, 4 , s rb, 5 } forming sentence pairs with it are filled with green, as shown in the table. With the collaboration of these two subtasks, we can perform the overall argument pair extraction task. In this case, two argument pairs (highlighted in blue/pink from two passages) are extracted, which correspond to the two green rectangles shown in the table. Figure 2 shows our proposed attention-guided multi-layer multi-cross (MLMC) encoding based model. The model mainly consists of three parts: a sentence embedder, an n -layer multi-cross encoder, and a predictor. The review sentences and rebuttal sentences first go through the sentence embedder separately to obtain their sentence embeddings respectively. We then utilize the representations from review and rebuttal sequences to form a table as shown earlier in Figure 1. Next, the representations of the table and two sequences are updated through n multi-cross encoder layers. Finally, the model predicts the review and rebuttal arguments through a conditional random field (CRF) (Laf-ferty et al., 2001) layer based on two sequence representations, and extracts the pairing information through a multi-layer perceptron (MLP) based on the table representations. The bottom left part of Figure 2 shows our sentence embedder, the input of which is a review sentence or a rebuttal sentence with l tokens s = [ t 0 , t 1 ,    , t l  1 ] . We obtain the pre-trained BERT (Devlin et al., 2019) token embeddings [ x 0 , x 1 ,    , x l  1 ] for all word tokens in the sentence, after which all token embeddings are fed into a bidirectional long short-term memory (biLSTM) BERT ... ... ... ... (Hochreiter and Schmidhuber, 1997) layer. The last hidden states from both directions are concatenated as the sentence embedding S (0) . A more common practice is to use the [CLS] token embedding to represent the sentence embedding. However, given the high density of scientific terms and the correspondence between review and rebuttal, token-level information is naturally crucial for the task. The same conclusion is drawn by the experimental results in the previous work (Cheng et al., 2020). The entire multi-cross encoder consists of n layers. The details of each multi-cross encoder layer are shown in the blue dotted box on the right of Figure 2. The input of the layer includes table representations and two sequence representations, i.e., review and rebuttal sequence representations. In each layer, table features are updated by sequence features and vice versa. Sequence Encoder Phase I To well-utilize different characteristics of review and rebuttal, we regard them as two individual sequences. Two sequence embeddings S ( k  1 ) rv and S ( k  1 ) rb of length I and J respectively (i.e., the output from the previous layer) are passed through the same biLSTM layer colored light yellow in Figure 2. Take review sequence as an example, the review hidden states at position i are updated as follows: S ( k ) (cid:48) [1] rv,i = LSTM forward ( S ( k  1) rv,i , S ( k ) (cid:48) [1] rv,i  1 ) , S ( k ) (cid:48) [2] rv,i = LSTM backward ( S ( k  1) rv,i , S ( k ) (cid:48) [2] rv,i +1 ) , S ( k ) (cid:48) rv,i = [ S ( k ) (cid:48) [1] rv,i , S ( k ) (cid:48) [2] rv,i ] . The rebuttal hidden states S ( k ) (cid:48) rb in layer k is obtained from the same biLSTM in the same manner. Table Encoder To capture the pairing information explicitly, we adopt the table-filling approach. At layer k , we update the table T ( k  1 ) rv  rb through the table encoder. The table input T ( 0 ) rv  rb before the first encoder layer are set as 0 . At each layer k , in order to incorporate the information extracted in S ( k ) (cid:48) rv and S ( k ) (cid:48) rb , we form another table T ( k  1 ) (cid:48)(cid:48) rv  rb with them through concatenation and linear projection as follows: T ( k  1 ) (cid:48)(cid:48) rv  rb = Linear ( S ( k ) (cid:48) rv  S ( k ) (cid:48) rb ) . The table features from previous layer T ( k  1 ) rv  rb are then updated by T ( k  1 ) (cid:48)(cid:48) rv  rb with layer normalization: T ( k  1 ) (cid:48) rv  rb = LayerNorm ( T ( k  1 ) rv  rb  T ( k  1 ) (cid:48)(cid:48) rv  rb ) . The entry T ( k  1) (cid:48) i,j at row i and column j represents specific features between review sentence at position i and rebuttal sentence at position j . The table hidden states T ( k ) i,j are updated through 2D-GRU: T ( k ) [1] i,j = GRU forward ( T ( k ) [1] i  1 ,j , T ( k ) [1] i,j  1 , T ( k  1) (cid:48) i,j ) , T ( k ) [2] i,j = GRU backward ( T ( k ) [2] i +1 ,j , T ( k ) [2] i,j +1 , T ( k  1) (cid:48) i,j ) , T ( k ) i,j = [ T ( k ) [1] i,j , T ( k ) [2] i,j ] . The 2D-GRU settings are similar to the previous work (Wang and Lu, 2020) except that the table to be processed is not necessarily a square ( I (cid:54) = J in general). Therefore, the 2D-GRU implemented here is more general. The previous hidden states for table boundaries ( T ( k ) [1] 0 ,j , T ( k ) [1] i, 0 , T ( k ) [2] I +1 ,j , T ( k ) [2] i,J +1 ) are set as 0 . The outputs T ( k ) rv  rp of layer k are further exploited by the mutual attention mechanism explained below to update review and rebuttal sequence embeddings. Mutual Attention The mutual attention mechanism (shown as review attention and rebuttal attention modules in Figure 2) links review embedding, rebuttal embedding and table embedding together, through which review embedding and rebuttal embedding update each other with the help of table features. The attention weights  ( k ) i,j and  ( k ) i,j at position ( i, j ) in layer k are updated as follows:  ( k ) i,j = tanh ( v T   T ( k ) i,j ) ,  ( k ) i,j = tanh ( v T   T ( k ) i,j ) , where v  and v  are learnable vectors. We further normalize the attention weights: a ( k ) i,j = exp(  ( k ) i,j ) (cid:80) Jj (cid:48) =1 exp(  ( k ) i,j (cid:48) ) , b ( k ) i,j = exp(  ( k ) i,j ) (cid:80) Ii (cid:48) =1 exp(  ( k ) i (cid:48) ,j ) . Here, a ( k ) i,j and b ( k ) i,j are the normalized attention weights ranging from 0 to 1. We then get the weighted average of sentence representations S ( k ) (cid:48)(cid:48) rv,i and S ( k ) (cid:48)(cid:48) rb,j from S ( k ) (cid:48) rb and S ( k ) (cid:48) rv respectively. S ( k ) (cid:48)(cid:48) rv,i = J (cid:80) j =1 a ( k ) i,j  S ( k ) (cid:48) rb,j , S ( k ) (cid:48)(cid:48) rb,j = I (cid:80) i =1 b ( k ) i,j  S ( k ) (cid:48) rv,i . Here, S ( k ) (cid:48)(cid:48) rv and S ( k ) (cid:48)(cid:48) rb are the updated review embedding and rebuttal embedding. Information in review and rebuttal sequences is exchanged via mutual attention. Sequence Encoder Phase II The addition and layer normalization used to combine S ( k ) (cid:48) and S ( k ) (cid:48)(cid:48) in the sequence encoder are similar to the one in table encoder. We obtain the review sequence embedding S ( k ) rv and rebuttal sequence embedding S ( k ) rb as the sequence outputs of layer k as follows: S ( k ) rv = LayerNorm ( S ( k ) (cid:48) rv  S ( k ) (cid:48)(cid:48) rv ) , S ( k ) rb = LayerNorm ( S ( k ) (cid:48) rb  S ( k ) (cid:48)(cid:48) rb ) . Stacking Multi-Cross Encoder Layers The updating process described above continues as layer grows from 1 to n . The table feature is updated by both review and rebuttal sequences, and each sequence updates the other via the table later on. There are also residual connections between adjacent layers which accept the previous layer's output as the current layer's input and include it as part of the new embedding, making the system more robust. All three features (i.e., review sequence, rebuttal sequence, table) are intertwined with each other and information flows across different components of the encoder. This is also the reason why the encoder is described as MLMC. After the final multi-cross encoder layer, sequence features are used for argument mining and table features are used for pair prediction. Argument Predictor We adopt CRF to predict argument sequence labels. The sequence labeling loss L seq for both review sequence s rv and rebuttal sequence s rb in each instance is defined as: L seq =  (cid:0) log p ( y rv | s rv ) + log p ( y rb | s rb ) (cid:1) , where y rv and y rb are the review and rebuttal sequence labels 2 . During inference, the predicted sequence label is the one with the highest conditional probability given the original sequence: y  rv = arg max y p ( y | s rv ) , y  rb = arg max y p ( y | s rb ) . Pair Predictor We use MLP to predict sentence pairs 3 . The pairing loss L pair for each instance is: L pair =  (cid:80) i,j (cid:16) y pairi,j log p ( y pairi,j = 1 | s rv , s rb ) + (1  y pairi,j ) log p ( y pairi,j = 0 | s rv , s rb ) (cid:17) , where y pairi,j is 1 when s rv,i and s rb,j are paired, and is 0 otherwise 4 . Following (Cheng et al., 2020), during evaluation, a pair of candidate spans ( [ s rv,i 1 ,    , s rv,i 2 ] and [ s rb,j 1 ,    , s rb,j 2 ] ) form a pair if they satisfy the following criterion: (cid:80) i 2 i = i 1 (cid:80) j 2 j = j 1 1 { p ( ypairi,j =1) > 0 . Attention Loss Attention loss is a loss term specifically designed for the task. It aims to increase the effectiveness of review attention and rebuttal attention discussed above. Even without 2 We provide the detailed steps of deriving the loss L seq in Appendix A.1. 3 MLP is chosen because more complex structures like convolutional neural networks (CNN) demonstrate no superiority. The comparison results are attached in Appendix B.3. 4 We provide the detailed steps of deriving the pairing loss L pair in Appendix A.2. this auxiliary loss term, sentences in review are supposed to attend to relevant sentences in rebuttal and vice versa. The auxiliary loss is thus aimed at augmenting the effect of mutual reference explicitly by guiding the paired arguments to refer to each other. Intuitively, under the settings of argument mining and pairing, it is natural that review arguments refer to the paired rebuttal arguments to update their embedding and vice versa during mutual attention. Hence, we introduce an auxiliary loss term to increase the attention weights computed for paired arguments and decrease the attention weights otherwise for both review and rebuttal attentions in all layers. For each instance, L attn is defined as: L attn = (cid:80) i,j (1  2 y pairi,j )  (cid:16) n (cid:80) k =1  n  k  ( a ( k ) i,j + b ( k ) i,j ) (cid:17) , where  is the decaying parameter used to compute exponential moving average for the sum of attention. Larger weights are assigned to layers closer to the final predictor as they are more related to the prediction in the end. The attention loss is defined in the form of summation across all layers to increase the accuracy and interpretability of both review and rebuttal attentions in all layers. If the tendency to attend to the paired argument is augmented, the benefits of attention mechanism can be further exploited (e.g., learning better sentence representations, increasing pair prediction accuracy). L = L seq +  1  L pair +  2  L attn , where  1 and  2 are tuned hyperparameters. We conduct experiments on the benchmark dataset, i.e., RR dataset (Cheng et al., 2020) to evaluate the effectiveness of our proposed model. RR dataset includes 4,764 pairs of peer reviews and author rebuttals collected from ICLR 2013 to ICLR 2020. There are two dataset versions provided: RR-Submission-v1 and RR-Passage-v1. In RR-Submission-v1, multiple review-rebuttal passage pairs of the same paper submission are in the same set of train, dev or test; while in RR-Passage-v1, different review-rebuttal passage pairs of the same submission could be put into different sets. We further modify the RR-Submission-v1 dataset by fixing some minor bugs in the labels, and name it RR-Submission-v2. The data are split into train, dev and test sets by a ratio of 8:1:1 for all three dataset versions. The pipeline approach is used as a baseline model in the previous work (Cheng et al., 2020). It independently trains two subtasks and then pipes them together to extract argument pairs. The multi-task learning model proposed by (Cheng et al., 2020) trains two subtasks simultaneously via the shared feature encoders. We implement our attention-guided MLMC encoding based model in Pytorch. The dimension of pre-trained BERT sentence embeddings is 768 by default. Maximum number of BERT tokens for each sentence is set as 200. MLP layer is composed of 3 linear functions and 2 ReLU functions. We use Adam (Kingma and Ba, 2014) with an initial learning rate of 0.0002, and update parameters with a batch size of 1 and dropout rate of 0.5. We train our model for 25 epochs at most. We select the best model parameters based on the best overall F 1 score on the development set and apply it to the test set for evaluation. All models are run with V100 GPU. Note that in this paper, the parameters are mainly tuned based on RR-Submission-v1 5 . Following the previous work (Cheng et al., 2020), we report the precision (Prec.), recall (Rec.) and F 1 scores for the performance on both subtasks as well as the overall extraction performance. Table 1 shows the performance comparison between our proposed models and the pervious work on RR-Submission-v1 and RR-Passage-v1 datasets 6 . Besides the two baseline models mentioned before, we implement a bi-cross encoding scheme (Bi-Cross) for comparisons as well. The key difference between the bi-cross encoder and the multi-cross encoder is that in the bi-cross encoder, 5 More details about hyperparameter settings (e.g. weight for pair loss  1 , weight for attention loss  2 , decaying parameter  of exponential moving average) and experimental results (e.g. running time, number of parameters, performance on the development set) could be found in Appendix B. 6 The previous work adopts negative sampling technique for sentence pairing subtask and evaluates the performance on the partial test set. In this work, we re-evaluate the previous work's sentence pairing subtask on the whole test dataset for a fair comparison. Those results are marked with * in Table 1. Data Models Argument Mining Sentence Pairing APE Prec. the review sentences and rebuttal sentences are concatenated as one sequence, and thus it only has one sequence encoder. In contrast, there are two individual sequence encoders in our multi-cross encoder. With the same number of layers, our multi-cross model outperforms the bi-cross model on both datasets except for RR-Passage-v1 with 4 layers. This is especially conspicuous when the number of layers is 3. The superiority of the multi-cross model demonstrates the importance and robustness of learning review and rebuttal sequences separately. Our model achieves the highest F 1 score when the number of layers increases to 3. Adding more layers hurts the performance, probably because the model overfits with too many layers. Table 2 shows the performance on RR-Submission-v2 7 . The main conclusion is consistent with the performace on RR-Submission-v1. Both the bi-cross and multi-cross models outperform the multitask model, and the multi-cross models further outperform the bi-cross models. Although the baselines achieve slightly better performance on the argument mining subtask than both the bi-cross model and the multi-cross model, they still perform worse than our models on the sentence pairing subtask and the overall APE task. This is plausibly because of two main reasons. First, in the multi-task model, the subtask coordi-7 We encourage the researchers to use RR-submission-v2 and compare to its performance in the future. nation capability is weak as the shared information between two subtasks is learned implicitly. However, in our model, the three encoding components are explicitly mingled with each other through the mutual attention mechanism and the table encoder. On one hand, the better sentence pairing subtask performance demonstrates the effectiveness of the table-filling approach. On the other hand, the better overall APE performance demonstrates the strong subtask coordination capability of our model architecture. Second, we further analyze the breakdown performance of the multi-task model and our multi-cross ( n =3) model on the argument mining subtask. Figure 3 shows the subtask performance on RR-Submission-v1 dataset for reviews, rebuttals, and both of them. We can observe that the difference of F 1 scores between reviews and rebuttals of our model is smaller than the multi-task model. Despite the slight decrease in the overall argument mining performance, a more balanced argument extraction performance on reviews and rebuttals brings in better overall APE performance, which is because more accurate review argument extraction increases the chance for the extracted rebuttal arguments to be paired correctly. We conduct an ablation study of the multi-cross ( n =3) model on RR-Submission-v1 dataset from three perspectives, as presented in Table 3. Firstly, we evaluate the effect of sharing the biLSTM layer (the light yellow modules in Figure 2) and the CRF layer. We can notice that the F 1 drops 1.92 without sharing the biLSTM layer, drops 1.75 without sharing the CRF layer, and drops 1.02 when sharing neither. It is interesting to notice that when two sequences use their own biLSTMs and CRF simultaneously (i.e., w/o sharing both), the F 1 drops less compared to the models without sharing only one of them. This suggests that having an individual set of biLSTM and CRF layers for each type of sequence is plausibly a worthwhile setting, but it Model Settings APE Prec. is not as effective as sharing both. One possible reason is that the advantage brought in by such a tailor-made sequential tagging configuration for each type is overwhelmed by the disadvantage of fewer training instances. Secondly, without cross updates between the review and rebuttal embeddings (the mutual attention modules still exist), the F 1 drops 1.78. This result again demonstrates the effectiveness of explicitly blending two sequence embeddings via the mutual attention mechanism specifically designed for this task. Thirdly, we also investigate the effect of attention loss term by removing it from the overall loss. The performance drops about 2.87 F 1 points. We will elaborate more with the attention visualization below. To examine the effectiveness of the auxiliary attention loss, we visualize the sum of attention weights of all layers for four test samples, as shown in Figure 4. The sum is computed for visualization because attention weights in all layers are guided by the attention loss. The distribution of attention is significantly improved as the colors for arguments in Column (c) are considerably darker. In Column (b) without the guidance of attention loss, despite some patterns, attention weights are distributed in a quite haphazard manner. Therefore, the interpretability of our model is much better as we can easily understand which part of the discourse each sentence refers to. Specifically, the boundary of most attention blocks in Column (c) matches well with the start and end positions of the ground truth review and rebuttal arguments. The gold and predicted argument spans and argument pairs of these four samples are shown in Appendix C.1, and more discussions are given regarding the reason for some mistakenly predicted boundaries. The effectiveness of the auxiliary attention loss is also quantitatively illustrated by a higher F 1 score after its incorporation (32.44 v.s. 29.57) in Table 3. OOOOOOBIIBIIBOO B I I I I I I I I O B I I O B I I I I I I I I O O O 1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627 1234567891011121314 OOOOOOBIIBIIBOO B I I I I I I I I O B I I O B I I I I I I I I O O O 1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627 1234567891011121314 OOOOOOBIIBIIBOO B I I I I I I I I O B I I O B I I I I I I I I O O O 1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627 1234567891011121314 OOOOOOBBIIIBIIBIIO O B I O B I I O B I I O B I I I I O O 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 1234567891011121314151617 OOOOOOBBIIIBIIBIIO O B I O B I I O B I I O B I I I I O O 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 1234567891011121314151617 OOOOOOBBIIIBIIBIIO O B I O B I I O B I I O B I I I I O O 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 1234567891011121314151617 OOOOOOOOBIBIIBIBBIBBIOOOOOOOB I OB I I OOB I B I I I I I I I I I I I I OOB I I B I OOB I I 1 2 3 4 5 6 7 8 91011121314151617181920212223242526272829303132333435363738 12345678910111213141516171819202122232425 OOOOOOOOBIBIIBIBBIBBIOOOOOOOB I OB I I OOB I B I I I I I I I I I I I I OOB I I B I OOB I I 1 2 3 4 5 6 7 8 91011121314151617181920212223242526272829303132333435363738 123456789 10111213141516171819202122232425 OOOOOOOOBIBIIBIBBIBBIOOOOOOOB I OB I I OOB I B I I I I I I I I I I I I OOB I I B I OOB I I 1 2 3 4 5 6 7 8 91011121314151617181920212223242526272829303132333435363738 12345678910111213141516171819202122232425 OOOOOOOBIBIIBBBBBBBIBO B I I I B I B I B B I B I I I B I O B I B O 1 2 3 4 5 6 7 8 9 1011121314151617181920212223 123456789101112131415161718192021 In this paper, we adopt the table-filling approach for modeling the sentence-level correlation between two passages, and propose the attention-guided multi-layer multi-cross (MLMC) encoding scheme for the argument pair extraction (APE) task. Our model can better capture the internal relations between a review and its rebuttal with two sequence encoders and a table encoder via mutual attention mechanism. We also introduce an auxiliary attention loss to further improve the efficacy of the mutual attentions. Extensive experiments on the benchmark dataset demonstrate the effectiveness of our model architecture, which is potentially ben-eficial for other NLP tasks.
abstain abstain objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain objective result abstain abstain method abstain objective objective objective result abstain other other other other other other other method other abstain abstain abstain other abstain abstain abstain abstain abstain method abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method other abstain method method abstain method abstain abstain other abstain abstain abstain abstain abstain other other abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain method abstain abstain method other other abstain abstain method abstain abstain method abstain abstain abstain method other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain result abstain result abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain other objective abstain result objective