Syntactic information, especially dependency trees, has been widely used by existing studies to improve relation extraction with better semantic guidance for analyzing the context information associated with the given entities. However, most existing studies suffer from the noise in the dependency trees, especially when they are automatically generated, so that intensively leveraging dependency information may introduce confusions to relation classification and necessary pruning is of great importance in this task. In this paper, we propose a dependency-driven approach for relation extraction with attentive graph convolutional networks (A-GCN). In this approach, an attention mechanism upon graph convolutional networks is applied to different contextual words in the dependency tree obtained from an off-the-shelf dependency parser, to distinguish the importance of different word dependencies. Consider that dependency types among words also contain important contextual guidance, which is potentially helpful for relation extraction, we also include the type information in A-GCN modeling. Experimental results on two English benchmark datasets demonstrate the effectiveness of our A-GCN, which outperforms previous studies and achieves state-of-the-art performance on both datasets. 1 1 Introduction Relation extraction (RE), which aims to detect the relationship between entity mentions from raw text, is one of the most important tasks in information extraction and retrieval, and plays a crucial role in supporting many downstream natural language processing (NLP) applications such as text mining (Distiawan et al., 2019), sentiment analysis (Sun * Equal contribution. and summarization (Wang and Cardie, 2012). Recently, neural RE methods (Zeng et al., 2014; Zhang and Wang, 2015; Xu et al., 2015; dos Santos et al., 2015; Zhang et al., 2015; Wang et al., 2016; Zhou et al., 2016; Zhang et al., 2017) with powerful encoders (such as CNN, RNN, and Transformers) have significantly improved model performance for RE without requiring any elaborately designed systems or manually constructed features. These methods are superior in capturing contextual information and thus enable RE systems to better understand the text and identify relations between entities in the given text. Adopting neural models to help RE is not only straightforward and effective, but is also expected to incorporate more diverse and informative knowledge into RE systems. Among all different knowledge sources, syntactic information, especially the dependency trees, have been demonstrated to be beneficial in many studies (Miwa and Bansal, 2016; Zhang et al., 2018; Sun et al., 2020; Chen et al., 2021) because they provide long-distance word connections between useful words and thus accordingly guide the system to better extract relations between entity pairs. However, intensively leveraging dependency information may not always lead to good RE performance, because the noise in the dependency tree can potentially introduce confusions to relation classification (Xu et al., 2015; Yu et al., 2020), especially when those trees are automatically generated. For example, Figure 1 shows an example sentence with its dependency tree, where the dependency connection between  pumpkin mixture  and  bowl  may introduce noise when the object is to predict the relation between  milk  and  pumpkin mixture . Therefore, previous studies have always required necessary pruning strategies before encoding the dependency information through a particular model such as LSTM (Xu et al., 2015) or graph convolutional networks (GCN) (Zhang et al., 2018). Because fixed pruning strategies are not guaranteed to result in a sub-tree with all important contextual information included and with all noise filtered out, it is necessary to design an appropriate way for distinguishing the noise in the dependency tree and modelling them accordingly. In this paper, we propose a dependency-driven neural approach for RE, where attentive graph neural network (A-GCN) is proposed to distinguish the important contextual information for this task. Furthermore, given that the dependency types (e.g., nominal subject) that associate with dependency connections are also potentially useful for RE since they contain the syntactic instruction among connected words, we further improve A-GCN by introducing type information into it. Specifically, we first obtain the dependency tree of an input sentence from an off-the-shelf toolkit, then build the graph over the dependency tree, and assign different weights to different labeled dependency connections between any two words, with the weights computed based on the connections and their dependency types, lastly predict relations by the A-GCN according to the learned weights. In doing so, not only is A-GCN able to distinguish important contextual information from dependency trees and leverage them accordingly, such that reliance on pruning strategies is unnecessary, but A-GCN can also leverage the dependency type information that is omitted by most previous studies (in particular, the studies that also use attention mechanism (Guo et al., 2019)). Experimental results on two English benchmark datasets, i.e., ACE2005EN and SemEval 2010 Task 8, demonstrate the effectiveness of our approach to RE through A-GCN equipped with dependency type information. State-of-the-art performance is observed on both datasets. by using A-GCN and incorporates dependency information to improve model performance, where the overall architecture of our model is illustrated in Figure 2. Specifically, given an unstructured input sentence X = x 1 ,    , x n with n words and let E 1 and E 2 denote two entities in X , our approach predicts the relation r between E 1 and E 2 by b 2 R where TX is the dependency tree of X obtained from an off-the-shelf toolkit, R is the relation type set; p computes the probability of a particular relation r 2 R given the two entities and b r the output of A-GCN, which takes X and TX as the input. Following texts start with a brief introduction of the standard GCN model, then elaborate our proposed A-GCN equipped with dependency type information, and lastly illustrate the process of applying A-GCN to the classification paradigm for RE. 2.1 Standard Graph Convolutional Networks Generally, a good text representation is a prerequisite to achieve outstanding model performance (Song et al., 2017; Bojanowski et al., 2017; Song et al., 2018; Song and Shi, 2018; Hajdik et al., 2019). To enhance the text representation and thus obtain a good understanding of the running text, many studies (Song et al., 2009, 2012; Song and Xia, 2013; Xu et al., 2015; Miwa and Bansal, 2016; Zhang et al., 2019; Mandya et al., 2020; Nie et al., 2020) tried to leverage contextual features, such as n-grams and syntactic information, through different model architectures. Among all these architecture choices, graph convolutional networks (GCN) is a widely used architecture to encode the information in a graph, where in each GCN layer, information in each node communicates to its neighbors through the connections between them. The effectiveness of GCN models to encode the contextual information over a graph of an input sentence has been demonstrated by many previous studies (Zhang et al., 2018; Guo et al., 2019; Sun et al., 2020; Chen et al., 2020; Yu et al., 2020; Mandya et al., 2020; Tian et al., 2020c, 2021a). Normally, the graph in the standard GCN model is built from word dependencies and is represented by an adjacency matrix A = ( a i,j ) n  n where a i,j = 1 if i = j or there is a dependency connection 2 (arc) between two words x i and x j in the dependency tree TX and a i,j = 0 otherwise. Based on A , for 2 Normally the direction of the connection is ignored. each word x i 2 X , the l -th GCN layer gathers the information carried by its context words in TX and computes the output representation h ( l ) i for x i by: h ( l ) i =  n X j =1 a i,j  W ( l )  h ( l  1) j + b ( l ) ! (2) where h ( l  1) j denotes the output representation of x j from the ( l 1) -th GCN layer 3 , W ( l ) and b ( l ) are trainable matrices and the bias for the l -th GCN layer, respectively, and  is the ReLU activation. It is noted that in standard GCN (e.g., Eq. (2)), the connections among words are treated equally (i.e., a i,j is either 0 or 1 ). Therefore, GCN-based models for RE are not able to distinguish the importance of different connections and thus pruning on them is of great importance for RE. Therefore, we propose A-GCN for this task, which uses an attention mechanism to compute the weights for different connections so that the model is able to 3 h (0) j is the output of the encoder for x j . leverage different dependency connections accordingly. In addition, the standard GCN and most previous studies omit the dependency types associated with the dependency connections, where those types contain highly useful information for RE and are introduced into A-GCN in this work. Specifi-cally, we firstly represent dependency types in TX by a type matrix T = ( t i,j ) n  n , where t i,j is the dependency type (e.g., nsubj ) associated with the directed dependency connection 4 between x i and x j . Next, we map each type t i,j to its embedding e ti,j . Then, at the l -th GCN layer, the weight for the connection between x i and x j is computed by p ( l ) i,j = a i,j  exp  s ( l ) i  s ( l ) j  P nj =1 a i,j  exp  s ( l ) i  s ( l ) j  (3) where a i,j 2 A ,    denotes inner production, and s ( l ) i and s ( l ) i are two intermediate vectors for x i and 4 It means t i,j and t j,i are represented in different dependency types to model directions of connections between x i and x j . x j , respectively, which are computed by s ( l ) i = h ( l  1) i  e ti,j (4) and s ( l ) j = h ( l  1) j  e ti,j (5) with    denoting the vector concatenation operation. Afterwards, we apply the weight p ( l ) i,j to the associated dependency connection between x i and x j and obtain the output representation of x i by h ( l ) i =  n X j =1 p ( l ) i,j  W ( l )  e h ( l  1) j + b ( l ) ! j Compared with standard GCN (i.e., Eq. (2)), our approach uses numerical weighting (i.e., p ( l ) i,j 2 [0 , 1] ) rather than a binary choice for a i,j , to distinguish the importance of different connections so as to leverage them accordingly. In addition, we integrate the dependency type information into both the computed weight (i.e., p ( l ) i,j ) and the output representation of x i (i.e., h ( l ) i ), which is not considered in most previous studies. Before applying A-GCN for RE, we firstly encode the input X into hidden vectors by BERT (Devlin et al., 2019) with h (0) i denoting the hidden vector for x i , where the hidden vector (denoted as h X ) for the special sentence initial token  [CLS]  is used as the representation for the entire sentence. Next, we feed h (0) i to our proposed A-GCN model with L layers and obtain the corresponding output h ( L ) i . Then, we apply the max pooling mechanism to the output hidden vectors of the words that belongs to an entity mention (i.e., E k , k = 1 , 2 ) to compute the representation for entity (denoted as h E k ) by h E k = MaxPooling ( { h ( L ) i | x i 2 E k } ) (8) Afterwards, we concatenate the representations of the sentence (i.e., h X ) and two entities (i.e., h E 1 and h E 2 ) and apply a trainable matrix WR to the computed vector to map it to the output space by o = WR  ( h X  h E 1  h E 2 ) (9) ACE05 SEMEVAL # INSTANCESTRAIN 48,198 8,000 DEV 11,854 TEST 10,097 2,717 Table 1: The number of unique instances (i.e., entity pairs) of ACE05 and SemEval benchmark datasets. where o is a |R| -dimensional vector with each of its value referring to a relation type in the relation type set R . Finally, we apply a softmax function of o to predict the relation b r between E 1 and E 2 by b r = arg max exp ( o u ) P |R| u =1 exp ( o u ) (10) with o u representing the value at dimension u in o . In the experiments, we use two English benchmark datasets for RE, namely, ACE2005EN (ACE05) 5 and SemEval 2010 Task 8 (SemEval) 6 (Hendrickx et al., 2010). For ACE05, we use its English section and follow previous studies (Miwa and Bansal, 2016; Christopoulou et al., 2018; Ye et al., 2019) to pre-process it (two small subsets cts and un are removed) and split the documents into training, development, and test sets 7 . For SemEval, we use its official train/test split 8 . The numbers of unique relation types in ACE05 and SemEval are 7 and 19, respectively. We report the number of instances (i.e., entity pairs), for train/dev/test sets of ACE05 and SemEval benchmark datasets in Table 1. 3.2 Dependency Graph Construction To construct graphs for A-GCN, we use Standard CoreNLP Toolkits (SCT) 9 to obtain the dependency tree TX for each input sentence X . Although our approach is able to distinguish the importance of different dependency connections through the attention mechanism, it is still beneficial if we can filter out those dependency connections that bring confusions to RE through particular pruning strategies. Motivated by previous studies (Xu et al., 2015; 5 We obtain the official data (LDC2006T06) from https: //catalog.ldc.upenn.edu/LDC2006T06 . 6 The data is downloaded from http://docs.google. com/View?docid=dfvxd49s_36c28v9pmw . 7 We follow the train/dev/test splits specified by Miwa and Bansal (2016) at https://github.com/tticoin/ LSTM-ER/tree/master/data/ace2005/split 8 SemEval only includes the training and test sets. 9 We download the version 3.9.2 from https:// stanfordnlp.github.io/CoreNLP/ . In detail, local connections include all dependencies that directly connect to the heads of two entities and global connections include all dependencies along the shortest dependency path (SDP) between the head of two entities, where in many cases words that do not directly connected to the two entities are also involved. With an example sentence including two entities (i.e.,  company  and benchmarking ), Figure 3 illustrates the two groups of dependency connections and the resulted adjacency matrix, which is built with the connections from the two groups 10 . It is worth noting that, when the SDP is short, there might be more connections in the local group than that in the global one. Following Soares et al. (2019), we insert four special tokens (i.e.,  < e1 > ,  < /e1 > ,  < e2 > , and  < /e2 > ) into the input sentence to mark the boundary 11 of the two entities to be investigated, which allows the encoder to distinguish the position of entities during encoding and thus improves model performance. For the encoder, we try BERT (Devlin et al., 2019), because it is a powerful pre-trained language model which and whose variants have achieved state-of-the-art performance in many NLP tasks (Wu and He, 2019; Soares et al., 2019; Wu et al., 2019; Diao et al., 2020; Song et al., 2020; Antoun et al., 2020; Tian et al., 2020a,b,d, 2021b; Qin et al., 2021; Song et al., 2021). Specifically, we use the uncased version of BERT-base and 10 We do not distinguish the two groups of connections in A-GCN once they are represented by the adjacency matrix. BERT-large 12 following the default settings (e.g., for BERT-base, we use 12 layers of multi-head attentions with 768-dimensional hidden vectors; for BERT-large, we use 24 layers of multi-head attentions with 1024-dimensional hidden vectors). For A-GCN, we randomly initialize all trainable parameters and the dependency type embeddings. For evaluation, we follow previous studies to use the standard micro-F1 scores 13 for ACE05 and use the macro-averaged F1 scores 14 for SemEval. In our experiments, we try different combinations of hyper-parameters, and tune them on the dev set, then evaluate on the test set by the model that achieves the highest F1 score on the dev set. 15 4 Results 4.1 Overall Results In the experiments, we run our A-GCN models using BERT-base and BERT-large encoder on graphs with and without applying dependency pruning strategies, which correspond to the graph built upon the combined local and global connections (L + G), as well as the one constructed by the full dependency graph (Full), respectively. We also run baselines with standard GCN and standard graph attentive networks (GAT) (Velickovic et al., 2017) with the same graph. For both standard GCN and A-GCN, we try different numbers of layers (i.e. 1 to 3 layers). In addition, we try BERT-base and BERT-large baselines without using any dependency information. Table 2 shows the F1 scores of our A-GCN 12 We download different BERT models from https:// github.com/huggingface/transformers . 13 We use the evaluation script from sklearn framework. 14 We use the official evaluation script downloaded from http://semeval2.fbk.eu/scorers/task08/SemEval2010_task8_scorer-v1.2.zip . 15 We report the hyper-parameter settings of different models with their size and running speed in Appendix A and B. ID MODELS ACE05 SEMEVAL 1 BERT-BASE 75.31 87.87 2 + GAT (FULL ) 76.16 88.39 3 + GAT (L + G) 75.79 88.53 4 + 1 GCNLAYER (FULL ) 74.91 87.58 5 + 1 A-GCNLAYER (FULL ) 76.63 88.34 6 + 1 GCNLAYER (L + G) 75.51 88.64 7 + 1 A-GCNLAYER (L + G) 77.10 89.03 8 + 2 GCNLAYERS (FULL ) 75.09 88.66 9 + 2 A-GCNLAYERS (FULL ) 77.25 88.70 10 + 2 GCNLAYERS (L + G) 76.11 88.62 11 + 2 A-GCNLAYERS (L + G) 77.30 89.16 12 + 3 GCNLAYERS (FULL ) 75.69 88.54 13 + 3 A-GCNLAYERS (FULL ) 76.26 88.63 14 + 3 GCNLAYERS (L + G) 76.85 88.33 15 + 3 A-GCNLAYERS (L + G) 76.36 88.70 (b) BERT-large Table 2: F1 scores of our A-GCN models and the baselines (i.e., BERT-only, standard GAT, and standard GCN) under different settings with BERT-base (a) and BERT-large (b) used. All graph-based models (i.e., GAT, GCN, and A-GCN) are tested with two settings: the first is using the full graph (FULL ) with all dependency connections involved and the second is using the combination of local and global connections (L + G). We also run GCN and A-GCN with different numbers of layers (i.e., 1 to 3 layers) for fair comparisons. models and all the aforementioned baselines on the test set of ACE05 and SemEval. 16 There are several observations. First, A-GCN functions well when using BERT-base or BERT-large as encoder, where the consistent improvement is observed over the BERT-only baselines (ID: 1) across two benchmark datasets, even though the BERT baselines have already achieve good performance. Second, for both datasets, A-GCN outperforms GAT (ID: 2, 3) and standard GCN baselines (ID: 4, 6, 8, 10, 12, 14) with the same graph (i.e., either L + G or Full) and equal number of layers. Particularly, when full dependency graph is used, it is noted that, in some cases (e.g., ID: 8 for BERT-base on ACE05), standard GCN obtains very limited improvements (or even worse results) over the BERT-only baseline (ID: 1), whereas our A-GCN models (e.g., ID: 9 for BERT-base) is able to consistently outperform the BERT-only baseline and achieve higher performance. We attribute this observation to the attention mechanism used to weigh different dependency connections, which allows A-GCN to distinguish the noise in the graph and thus leverage useful dependency information accordingly. Third, among the models with different numbers of A-GCN layers, the ones (e.g., ID: 11 for BERT-base and ID: 11 for BERT-large) with two A-GCN layers achieves the highest scores, where similar tread is observed from the standard GCN baselines. Besides, we find that our A-GCN 16 For the same group of models, we report the F1 scores on the development sets in Appendix C and the mean and standard deviation of their test set results in Appendix D. models (as well as the standard GCN baselines) with the local and global connections (i.e., L + G) consistently outperform the ones with full dependency graph (i.e., Full). These observations are relatively intuitive since the dependency information may introduce more noise to RE when it is leveraged in an intensive way (e.g., by using more layers or the full dependency tree without pruning). In addition, we compare our best models (with L + G or Full graphs) using BERT-large encoder and two A-GCN layers (ID: 9 and 11) with previous studies. The test results (F1 scores) are reported in Table 3, where our model with both local and global connections (i.e., L + G) outperforms all previous studies and achieves state-of-the-art performance on the two benchmark datasets. Specifically, compared with Guo et al. (2019) who proposed an graph-based approach with attentions to leverage dependency connections, our approach leverages both dependency connections and dependency types among all input words and thus provides a better way to comprehensively leverage the dependency information. In addition, although Mandya et al. (2020) proposed an approach to leverage both dependency connections and dependency types through attentions, they added the dependency type directly to the input word embeddings along with POS embeddings, and the attention in their approach is a separate stand-alone module which is added on the top of the GCN layer. On the contrary, in our approach, the dependency type MODELS ACE05 SEMEVALXU ET AL . is added to each A-GCN layer and the attention mechanism is directly applied to each dependency connection in the A-GCN layer. Therefore, compared with Mandya et al. (2020), our A-GCN encodes the dependency connections and dependency types in a more intensive manner and thus can better leverage them to guide the process of predicting the relations between the given entities. Dependency information is supposed to be beneficial for RE because it contains long-distance word-word relations, which could be extremely useful when the given two entities are far away from each other in the input sentence. To explore the effect of A-GCN in capturing such long-distance word-word relations to help with RE, we split the test instances into different groups according to their entities' distances (i.e., the number of words between the two entities) and run models on these groups to test their performance. Figure 4 shows the performance of our best performing A-GCN model with BERT-large (ID: 11 in Table 2) and its corresponding standard GCN and BERT-large baselines on the three groups of test instances from the test set of SemEval, where the category name indicates the range of the entity distance. 17 It is observed that, A-GCN outperforms the two baselines on all groups of test instances and the improvement becomes larger when the entity distance increases. This observation confirms that our approach is able to leverage dependency information and capture long-distance word-word relations to improve RE. 5.2 The Effect of Graph Construction In the main experiments, we try A-GCN with the graph built upon the combined local and global connections (L + G). To explore the effect of the local connections and the global connections for A-GCN, we run our approach using two A-GCN layers with the graph constructed by local connections (L) or global connections (G) alone. Table 4 presents the experimental results (F1 scores) of different models with BERT-base and BERT-large encoders, where the results from BERT-only baselines, A-GCN (L + G), and A-GCN (Full) are also copied from Table 2 for reference. Compared to A-GCN (L + G), models with the graph constructed by either local connections (i.e., A-GCN (L)) or global connections (i.e., A-GCN (G)) achieve lower performance, which complies with our intuition because both groups of connections contain important contextual features for RE. Interestingly, it is found that A-GCN (L) outperforms A-GCN (G) with both BERT-base and BERT-large encoders. A possible explanation could be the following. There are overlaps between local and global connections (e.g., the connection between  range  and  restrictions  in Figure 3). Therefore, A-GCN (L) can not only leverage the contextual information associated with the entities themselves, but is also partially 18 bene-fited from the overlapping connections on the SDP between the two entities, which leads A-GCN (L) to achieve a higher performance than A-GCN (G). Compared with the standard GCN, A-GCN enhances it from two aspects: (1) using an attention 18 When there is only one word on the shortest dependency path between two entities, all global connections are included in local ones, e.g.,  defamation  and  bishop  in Figure 2. mechanism to weigh different dependency connections and (2) introducing dependency types to the process to encode more detailed dependency information. To better investigate the effect of each individual enhancement (i.e., the attention mechanism or the dependency type information), we conduct an ablation study on our best model, i.e., two layers of A-GCN (L + G) with BERT-base and BERT-large encoder. Table 5 reports the experimental results of different models, where the performance of BERT-only baseline and the standard GCN baseline (i.e., the one uses neither the attention mechanism nor dependency types) are also reported for reference. The results clearly indicate that, the ablation of either enhancement (i.e., the attention mechanism or the dependency type information) could result in worse results (compared with full A-GCN). Between the two enhancements, the ablation of the attention mechanism hurts A-GCN more, which indicates the ability of distinguishing important connections and leveraging them accordingly plays a more important role in RE. To explore in detail that how A-GCN leverages dependency connections and types to improve RE, we conduct a case study with our A-GCN models with different dependency graphs (i.e., two layers of A-GCN (Full) and A-GCN (L + G) with BERT-large encoder) on an example sentence  A central vacuum is a vacuum motor and filtration system built inside a canister. . Figure 5 shows the sentence where both the two models correctly predict the relation between  motor  ( E 1 ) and  canister  ATT . ( E 2 ) (highlighted in the red color) to be  Content-Container , whereas the baseline GCN (Full) and GCN (L + G) models fail to do so. We also visualize the attention weights assigned to different dependency connections extracted from the last A-GCN layer, with darker and thicker lines referring to higher weights. In this example, for A-GCN (Full), we observe that the connection between  built  and  canister  along SDP and the connection between  inside  and  canister  receive the highest weights, where this is valid because the dependency type, i.e., obl (oblique nominal), associated with the connection (between  built  and  canister ) reveals that  canister  could be the position where the action (i.e., build ) takes place, and is further confirmed by another dependency connection and type (i.e., case ) between  inside  and  canister . Therefore, it is proved that our model learn from the contextual information carried by such important connections and results in correct RE prediction. Similarly, A-GCN (L + G) also correctly perform RE on this case by highlighting the same dependency connections as those from the A-GCN (Full) with much higher weights (because many dependency connections are filtered out). Recently, neural networks with integrating external knowledge or resources play important roles in RE because of their superiority in better capturing contextual information (Shen and Huang, 2016; Soares et al., 2019). Particularly, as one kind of such knowledge, dependency parses show their effectiveness in supporting RE for its ability Figure 5: Visualizations of weights assigned to different dependency connections of A-GCN (Full) and A-GCN (L + G) for an example input, where darker and thicker lines refer to connections with higher weights. in capturing long-distance word relations (Zhang et al., 2018; Guo et al., 2019). However, intensively leveraging dependency information could introduce confusions to RE (Xu et al., 2016b; Yu et al., 2020) so that necessary pruning is required to alleviate this problem. E.g., Xu et al. (2015) proposed to use the connections along the shortest dependency path between the two entities and apply LSTM to model them; Miwa and Bansal (2016) proposed to prune the original dependency tree into the lowest common ancestor subtree. However, these pruning strategies are either too aggressive or modest, so that the resulted graph might lose some important contexts or filled with more noise. Zhang et al. (2018) adopted GCN to model the dependencies and proposed a trade-off pruning strategy in between Xu et al. (2015) and Miwa and Bansal (2016). Besides, there are other graph-based models for RE that utilize layers of multihead attentions (Guo et al., 2019), dynamic pruning (Yu et al., 2020), and additional attention layers (Mandya et al., 2020) to encode dependency trees. Compared with the aforementioned methods, especially the graph-based ones, our approach offers an alternative to enhance RE with A-GCN by using attention mechanism and dependency type, which are effective and efficient improvement to standard GCN without requiring complicated model design. In this paper, we propose A-GCN to leverage dependency information for relation extraction, where an attention mechanism is applied to dependency connections to applying weighting on both connections and types so as to better distinguish the important dependency information and leverage them accordingly. In doing so, A-GCN is able to dynamically learn from different dependency connections so that less-informative dependencies are smartly pruned. Experimental results and analyses on two English benchmark datasets for relation extraction demonstrate the effectiveness of our approach, especially for entities with long word-sequence distances, where state-of-the-art performance is obtained on both datasets. This work is supported by Chinese Key-Area Research and Development Program of Guangdong Province (2020B0101350001) and NSFC under the project The Essential Algorithms and Technologies for Standardized Analytics of Clinical Texts (12026610). This work is also partially supported by Shenzhen Institute of Artificial Intelligence and Robotics for Society under the project Automatic Knowledge Enhanced Natural Language Understanding and Its Applications (AC01202101001). We also thank Mr. Peilin Zhou for providing the first version of the model architecture figure.
abstain abstain objective abstain method objective abstain abstain other abstain abstain abstain abstain abstain abstain abstain objective result objective abstain abstain abstain method abstain method abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method method method other abstain method abstain abstain method method method method method abstain method abstain abstain abstain abstain result abstain other other abstain other other abstain abstain abstain abstain result abstain abstain abstain abstain result result abstain abstain abstain other abstain other other abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain result abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain method other abstain abstain other other other other other other other other method objective abstain objective other other other