Most work in relation extraction forms a prediction by looking at a short span of text within a single sentence containing a single entity pair mention. This approach often does not consider interactions across mentions, requires redundant computation for each mention pair, and ignores relationships expressed across sentence boundaries. These problems are exacerbated by the document(rather than sentence-) level annotation common in biological text. In response, we propose a model which simultaneously predicts relationships between all mention pairs in a document. We form pairwise predictions over entire paper abstracts using an ecient self-attention encoder. All-pairs mention scores allow us to perform multi-instance learning by aggregating over mentions to form entity pair representations. We further adapt to settings without mention-level annotation by jointly training to predict named entities and adding a corpus of weakly labeled data. In experiments on two Biocreative benchmark datasets, we achieve state of the art performance on the Biocreative V Chemical Disease Relation dataset for models without external KB resources. We also introduce a new dataset an order of magnitude larger than existing human-annotated biological information extraction datasets and more accurate than distantly supervised alternatives. With few exceptions (Swampillai and Stevenson, 2011; Quirk and Poon, 2017; Peng et al., 2017), nearly all work in relation extraction focuses on classifying a short span of text within a single sentence containing a single entity pair mention. However, relationships between entities are often expressed across sentence boundaries or otherwise require a larger context to disambiguate. For example, 30% of relations in the Biocreative V CDR dataset (3.1) are expressed across sentence boundaries, such as in the following excerpt expressing a relationship between the chemical azathioprine and the disease fibrosis : Treatment of psoriasis with azathioprine . Azathioprine treatment benefited 19 (66%) out of 29 patients suering from severe psoriasis. Haematological complications were not troublesome and results of biochemical liver function tests remained normal. Minimal cholestasis was seen in two cases and portal fibrosis of a reversible degree in eight. Liver biopsies should be undertaken at regular intervals if azathioprine therapy is continued so that structural liver damage may be detected at an early and reversible stage. Though the entities' mentions never occur in the same sentence, the above example expresses that the chemical entity azathioprine can cause the side eect fibrosis . Relation extraction models which consider only within-sentence relation pairs cannot extract this fact without knowledge of the complicated coreference relationship between eight and azathioprine treatment , which, without features from a complicated pre-processing pipeline, cannot be learned by a model which considers entity pairs in isolation. Making separate predictions for each mention pair also obstructs multi-instance learning (Riedel et al., 2010; Surdeanu et al., 2012), a technique which aggregates entity representations from mentions in order to improve robustness to noise in the data. Like the majority of relation extraction data, most annotation for biological relations is distantly supervised, and so we could benefit from a model which is amenable to multi-instance learning. In addition to this loss of cross-sentence and cross-mention reasoning capability, traditional mention pair relation extraction models typically introduce computational ineciencies by independently extracting features for and scoring every pair of mentions, even when those mentions occur in the same sentence and thus could share representations. In the CDR training set, this requires separately encoding and classifying each of the 5,318 candidate mention pairs independently, versus encoding each of the 500 abstracts once. Though abstracts 872 are longer than e.g. the text between mentions, many sentences contain multiple mentions, leading to redundant computation. However, encoding long sequences in a way which eectively incorporates long-distance context can be prohibitively expensive. Long Short Term Memory (LSTM) networks (Hochreiter and Schmidhuber, 1997) are among the most popular token encoders due to their capacity to learn high-quality representations of text, but their ability to leverage the fastest computing hardware is thwarted due to their computational dependence on the length of the sequence  each token's representation requires as input the representation of the previous token, limiting the extent to which computation can be parallelized. Convolutional neural networks (CNNs), in contrast, can be executed entirely in parallel across the sequence, but the amount of context incorporated into a single token's representation is limited by the depth of the network, and very deep networks can be dicult to learn (Hochreiter, 1998). These problems are exacerbated by longer sequences, limiting the extent to which previous work explored full-abstract relation extraction. To facilitate ecient full-abstract relation extraction from biological text, we propose Bi-ane Relation Attention Networks (BRANs), a combination of network architecture, multi-instance and multi-task learning designed to extract relations between entities in biological text without requiring explicit mention-level annotation. We synthesize convolutions and self-attention, a modification of the Transformer encoder introduced by Vaswani et al. (2017), over sub-word tokens to eciently incorporate into token representations rich context between distant mention pairs across the entire abstract. We score all pairs of mentions in parallel using a bi-ane operator, and aggregate over mention pairs using a soft approximation of the max function in order to perform multi-instance learning. We jointly train the model to predict relations and entities, further improving robustness to noise and lack of gold annotation at the mention level. In extensive experiments on two benchmark biological relation extraction datasets, we achieve state of the art performance for a model using no external knowledge base resources in experiments on the Biocreative V CDR dataset, and outperform comparable baselines on the Biocreative VI ChemProt dataset. We also introduce a new dataset which is an order of magnitude larger than existing gold-annotated biological relation extraction datasets while covering a wider range of entity and relation types and with higher accuracy than distantly supervised datasets of the same size. We provide a strong baseline on this new dataset, and encourage its use as a benchmark for future biological relation extraction systems. We designed our model to eciently encode long contexts spanning multiple sentences while forming pairwise predictions without the need for mention pair-specific features. To do this, our model first encodes input token embeddings using self-attention. These embeddings are used to predict both entities and relations. The relation extraction module converts each token to a head and tail representation. These representations are used to form mention pair predictions using a bi-ane operation with respect to learned relation embeddings. Finally, these mention pair predictions are pooled to form entity pair predictions, expressing whether each relation type is expressed by each relation pair. Our model takes in a sequence of N token embeddings in R d . Because the Transformer has no innate notion of token position, the model relies on positional embeddings which are added to the input token embeddings. 2 We learn the position embedding matrix P m  d which contains a separate d dimensional embedding for each position, limited to m possible positions. Our final input representation for token x i is: x i = s i + p i where s i is the token embedding for x i and p i is the positional embedding for the i th position. If i exceeds m , we use a randomly initialized vector in place of p i . We tokenize the text using byte pair encoding (BPE) (Gage, 1994; Sennrich et al., 2015). The BPE algorithm constructs a vocabulary of sub-word pieces, beginning with single characters. Then, the algorithm iteratively merges the most frequent co-occurring tokens into a new token, which is added to the vocabulary. This procedure continues until a pre-defined vocabulary size is met. BPE is well suited for biological data for the following reasons. First, biological entities often have unique mentions made up of meaningful subcomponents, such as 1,2-dimethylhydrazine . Additionally, tokenization of chemical entities is challenging, lacking a universally agreed upon algorithm (Krallinger et al., 2015). As we demonstrate in 3.3.2, the subword representations produced by BPE allow the model to formulate better predictions, likely due to better modeling of rare and unknown words. Though our final model incorporates some convolutions, we retain the position embeddings. Transformer is made up of B blocks. Each Transformer block, which we denote Transformer k , has its own set of parameters and is made up of two subcomponents: multi-head attention and a series of convolutions 3 . The output for token i of block k , b ( k ) i , is connected to its input b ( k  1) i with a residual connection (He et al., 2016). Starting with b (0) i = x i : b ( k ) i = b ( k  1) i + Transformer k ( b ( k  1) i ) 2.2.1 Multi-head Attention Multi-head attention applies self-attention multiple times over the same inputs using separately normalized parameters (attention heads) and combines the results, as an alternative to applying one pass of attention with more parameters. The intuition behind this modeling decision is that dividing the attention into multiple heads make it easier for the model to learn to attend to dierent types of relevant information with each head. The self-attention updates input b ( k  1) i by performing a weighted sum over all tokens in the sequence, weighted by their importance for modeling token i . Each input is projected to a key k , value v , and query q , using separate ane transformations with ReLU activations (Glorot et al., 2011). Here, k , v , and q are each in R dH where H is the number of heads. The attention weights a ijh for head h between tokens i and j are computed using scaled dot-product attention: a ijh =  (cid:18) q Tih k jh  d (cid:19) o ih = X j v jh (cid:12) a ijh with (cid:12) denoting element-wise multiplication and  indicating a softmax along the j th dimension. The scaled attention is meant to aid optimization by flattening the softmax and better distributing the gradients (Vaswani et al., 2017). The outputs of the individual attention heads are concatenated, denoted [  ;  ] , into o i . All layers in the network use residual connections between the output of the multi-headed attention and its input. Layer normalization (Ba et al., 2016), denoted LN (  ) , is then applied to the output. The second part of our Transformer block is a stack of convolutional layers. The sub-network used in 3 The original Transformer uses feed-forward connections, i.e. width-1 convolutions, whereas we use convolutions with width > 1. Vaswani et al. (2017) uses two width-1 convolutions. We add a third middle layer with kernel width 5, which we found to perform better. Many relations are expressed concisely by the immediate local context, e.g. Michele's husband Barack , or labetalol-induced hypotension . Adding this explicit n-gram modeling is meant to ease the burden on the model to learn to attend to local features. We use C w (  ) to denote a convolutional operator with kernel width w . Then the convolutional portion of the transformer block is given by: t (0) i = ReLU( C 1 ( m i )) t (1) i = ReLU( C 5 ( t (0) i )) t (2) i = C 1 ( t (1) i ) Where the dimensions of t (0) i and t (1) i are in R 4 d and that of t (2) i is in R d . We project each contextually encoded token b ( B ) i through two separate MLPs to generate two new versions of each token corresponding to whether it will serve as the first (head) or second (tail) argument of a relation: where L is a d  L  d tensor, a learned embedding matrix for each of the L relations. In subsequent sections we will assume we have transposed the dimensions of A as d  d  L for ease of indexing. Our data is weakly labeled in that there are labels at the entity level but not the mention level, making the problem a form of strong-distant supervision (Mintz et al., 2009). In distant supervision, edges in a knowledge graph are heuristically applied to sentences in an auxiliary unstructured text corpus  often applying the edge label to all sentences containing the subject and object of the relation. Because this process is imprecise and introduces noise into the training data, methods like multi-instance learning were introduced (Riedel et al., 2010; Surdeanu et al., 2012). In multi-instance learning, rather than looking at each distantly labeled mention pair in isolation, the model is trained over the aggregate of these mentions and a single update is made. More recently, the weighting function of the instances has been expressed as neural network attention (Verga and McCallum, 2016; Lin et al., 2016; Yaghoobzadeh et al., 2017). We aggregate over all representations for each mention pair in order to produce per-relation scores for each entity pair. For each entity pair ( p head , p tail ) , let P head denote the set of indices of mentions of the entity p head , and let P tail denote the indices of mentions of the entity p tail . Then we use the LogSumExp function to aggregate the relation scores from A across all pairs of mentions of p head and p tail : scores ( p head , p tail ) = log X i  P head j  P tail exp( A ij ) The LogSumExp scoring function is a smooth approximation to the max function and has the bene-fits of aggregating information from multiple predictions and propagating dense gradients as opposed to the sparse gradient updates of the max (Das et al., 2017). In addition to pairwise relation predictions, we use the Transformer output b ( B ) i to make entity type predictions. We feed b ( B ) i as input to a linear classifier which predicts the entity label for each token with per-class scores c i : c i = W (3) b ( B ) i We augment the entity type labels with the BIO encoding to denote entity spans. We apply tags to the byte-pair tokenization by treating each subword within a mention span as an additional token with a corresponding Bor Ilabel. We train both the NER and relation extraction components of our network to perform multi-class classification using maximum likelihood, where NER classes y i or relation classes r i are conditionally independent given deep features produced by our model with probabilities given by the softmax function. In the case of NER, features are given by the per-token output of the transformer: 1 NNX i =1 log P ( y i | b ( B ) i ) In the case of relation extraction, the features for each entity pair are given by the LogSumExp over pairwise scores described in  2.4. For E entity pairs, the relation r i is given by: 1 EEX i =1 log P ( r i | scores ( p head , p tail )) 875 We train the NER and relation objectives jointly, sharing all embeddings and Transformer parameters. To trade o the two objectives, we penalize the named entity updates with a hyperparameter  . We evaluate our model on three datasets: The Biocreative V Chemical Disease Relation benchmark (CDR), which models relations between chemicals and diseases (3.1); the Biocreative VI ChemProt benchmark (CPR), which models relations between chemicals and proteins (3.2); and a new, large and accurate dataset we describe in 3.3 based on the human curation in the Chemical Toxicology Database (CTD), which models relationships between chemicals, proteins and genes. The CDR dataset is annotated at the level of paper abstracts, requiring consideration of long-range, cross sentence relationships, thus evaluation on this dataset demonstrates that our model is capable of such reasoning. We also evaluate our model's performance in the more traditional setting which does not require cross-sentence modeling by performing experiments on the CPR dataset, for which all annotations are between two entity mentions in a single sentence. Finally, we present a new dataset constructed using strong-distant supervision (2.4), with annotations at the document level. This dataset is significantly larger than the others, contains more relation types, and requires reasoning across sentences. The Biocreative V chemical disease relation extraction (CDR) dataset 4 (Li et al., 2016a; Wei et al., 2016) was derived from the Comparative Toxicogenomics Database (CTD), which curates interactions between genes, chemicals, and diseases (Davis et al., 2008). CTD annotations are only at the document level and do not contain mention annotations. The CDR dataset is a subset of these original annotations, supplemented with human annotated, entity linked mention annotations. The relation annotations in this dataset are also at the document level only. The CDR dataset is concerned with extracting only chemically-induced disease relationships (drug-related side eects and adverse reactions) concerning the most specific entity in the document. For example tobacco causes cancer could be marked as false if the document contained the more specific lung cancer . This can cause true relations to be labeled as false, harming evaluation performance. To address this we follow (Gu et al., 2016, 2017) 4 http://www.biocreative.org/ and filter hypernyms according to the hierarchy in the MESH controlled vocabulary 5 . All entity pairs within the same abstract that do not have an annotated relation are assigned the NULL label. In addition to the gold CDR data, Peng et al. (2016) add 15,448 PubMed abstracts annotated in the CTD dataset. We consider this same set of abstracts as additional training data (which we subsequently denote +Data). Since this data does not contain entity annotations, we take the annotations from Pubtator (Wei et al., 2013), a state of the art biological named entity tagger and entity linker. See A.1 for additional data processing details. In our experiments we only evaluate our relation extraction performance and all models (in-cluding baselines) use gold entity annotations for predictions. The byte pair vocabulary is generated over the training dataset  we use a budget of 2500 tokens when training on the gold CDR data, and a larger budget of 10,000 tokens when including extra data described above Additional implementation details are included in Appendix A. Data split Docs Pos Neg Train 500 1,038 4,280 Development 500 1,012 4,136 Test 500 1,066 4,270 CTD 15,448 26,657 146,057 Table 1: Data statistics for the CDR Dataset and additional data from CTD. We compare against the previous best reported results on this dataset not using knowledge base features. 6 Each of the baselines are ensemble methods for withinand cross-sentence relations that make use of additional linguistic features (syntactic parse and part-of-speech). Gu et al. (2017) encode mention pairs using a CNN while Zhou et al. (2016a) use an LSTM. Both make cross-sentence predictions with featurized classifiers. In Table 2 we show results outperforming the baselines despite using no linguistic features. We show performance averaged over 20 runs with 20 random seeds as well as an ensemble of their averaged predictions. We see a further boost in performance by adding weakly labeled data. Table 3 shows the 5 https://www.nlm.nih.gov/mesh/download/ 2017MeshTree.txt 6 The highest reported score is from (Peng et al., 2016), but they use explicit lookups into the CTD knowledge base for the existence of the test entity pair. eects of ablating pieces of our model. CNN only' removes the multi-head attention component from the transformer block, no width-5' replaces the width-5 convolution of the feed-forward component of the transformer with a width-1 convolution and no NER' removes the named entity recognition multi-task objective (2.5). To assess our model's performance in settings where cross-sentence relationships are not explicitly evaluated, we perform experiments on the Biocreative VI ChemProt dataset (CDR) (Krallinger et al., 2017). This dataset is concerned with classifying into six relation types between chemicals and proteins, with nearly all annotated relationships occurring within the same sentence. We compare our models against those competing in the ocial Biocreative VI competition (Liu et al., 2017). We compare to the top performing team whose model is directly comparable with ours  i.e. used a single (non-ensemble) model trained only on the training data (many teams use the development set as additional training data). The baseline models are standard state of the art relation extraction models: CNNs and Gated RNNs with attention. Each of these baselines uses mention-specific features encoding relative position of each token to the two target entities being classified, whereas our model aggregates over all mention pairs in each sentence. It is also worth noting that these models use a large vocabulary of pre-trained word embeddings, giving their models the advantage of far more model parameters, as well as additional information from Model P R F1 CNN  50.7 43.0 46.5 GRU+Attention  53.0 46.3 49.5 BRAN 48.0 54.1 50.8  .01 Table 4: Precision, recall, and F1 results on the Biocreative VI Chem-Prot Dataset. In Table 4 we see that even though our model forms all predictions simultaneously between all pairs of entities within the sentence, we are able to outperform state of the art models classifying each mention pair independently. The scores shown are averaged across 10 runs with 10 random seeds. Interestingly, our model appears to have higher recall and lower precision, while the baseline models are both precision-biased, with lower recall. This suggests that combining these styles of model could lead to further gains on this task. Existing biological relation extraction datasets including both CDR (3.1) and CPR (3.2) are relatively small, typically consisting of hundreds or a few thousand annotated examples. Distant supervision datasets apply document-independent, entity-level annotations to all sentences leading to a large proportion of incorrect labels. Evaluations on this data involve either very small (a few hundred) gold annotated examples or cross validation to predict the noisy, distantly applied labels (Mallory et al., 2015; Quirk and Poon, 2017; Peng et al., 2017). We address these issues by constructing a new dataset using strong-distant supervision containing document-level annotations. The Comparative Toxicogenomics Database (CTD) curates interactions between genes, chemicals, and diseases. Each relation in the CTD is associated with a disambiguated entity pair and a PubMed article where the relation was observed. To construct this dataset, we collect the abstracts for each of the PubMed articles with at least one curated relation in the CTD database. As in 3.1, we use PubTator to automatically tag and disambiguate the entities in each of these abstracts. If both entities in the relation are found in the abstract, we take the (abstract, relation) pair as a positive example. The evidence for the curated relation could occur anywhere in the full text article, not just the abstract. Abstracts with no recovered relations are discarded. All other entity pairs with valid types and without an annotated relation that 877 Types Docs Pos Neg Total 68,400 166,474 1198,493 Chemical/Disease 64,139 93,940 571,932 Chemical/Gene 34,883 63,463 360,100 Gene/Disease 32,286 9,071 266,461 Table 5: Data statistics for the new CTD dataset. occur in the remaining abstracts are considered negative examples and assigned the NULL label. We additionally remove abstracts containing greater than 500 tokens 7 . This limit removed about 10% of the total data including numerous extremely long abstracts. The average token length of the remaining data was 230 tokens. With this procedure, we are able to collect 166,474 positive examples over 13 relation types, with more detailed statistics of the dataset listed in Table 5. We consider relations between chemical-disease, chemical-gene, and gene-disease entity pairs downloaded from CTD 8 . We remove inferred relations (those without an associated PubMed ID) and consider only human curated relationships. Some chemical-gene entity pairs were associated with multiple relation types in the same document. We consider each of these relation types as a separate positive example. The chemical-gene relation data contains over 100 types organized in a shallow hierarchy. Many of these types are extremely infrequent, so we map all relations to the highest parent in the hierarchy, resulting in 13 relation types. Most of these chemical-gene relations have an increase and decrease version such as increase_expression and de-crease_expression. In some cases, there is also an aects relation (aects_expression) which is used when the directionality is unknown. If the aects version is more common, we map decrease and increase to aects. If aects is less common, we drop the aects examples and keep the increase and decrease examples as distinct relations, resulting in the final set of 10 chemical-gene relation types. In Table 7 we list precision, recall and F1 achieved by our model on the CTD dataset, both overall and by relation type. Our model predicts each of the relation types eectively, with higher performance on relations with more support. In Table 8 we see that our sub-word BPE model out-performs the model using the Genia tokenizer (Kulick et al., 2012) even though our vocabulary size is one-fifth as large. We see a 1.7 F1 point boost in predicting Pubtator NER labels for BPE. This could be explained by the increased out-of-7 We include scripts to generate the unfiltered set of data as well to encourage future research 8 http://ctdbase.org/downloads/ Train Dev Test Total 120k 15k 15k Chemical/Disease marker/mechanism 41,562 5,126 5,167 therapeutic 24,151 2,929 3,059 Gene/Disease marker/mechanism 5,930 825 819 therapeutic 560 77 75 Chemical/Gene increase_expression 15,851 1,958 2,137 increase_MP 5,986 740 638 decrease_expression 5,870 698 783 increase_activity 4,154 467 497 aects_response 3,834 475 508 decrease_activity 3,124 396 434 aects_transport 3,009 333 361 increase_reaction 2,881 367 353 decrease_reaction 2,221 247 269 decrease_MP 798 100 120 Table 6: Data statistics for the new CTD dataset broken down by relation type. vocabulary (OOV) rate for named entities. Word training data has 3.01 percent OOV rate for tokens with an entity. The byte pair-encoded data has an OOV rate of 2.48 percent. Note that in both the word-tokenized and byte pair-tokenized data, we replace tokens that occur less than five times with a learned UNK token. Figure 2 depicts the model's performance on relation extraction as a function of distance between entities. For example, the blue bar depicts performance when removing all entity pair candidates (positive and negative) whose closest mentions are more than 11 tokens apart. We consider removing entity pair candidates with distances of 11, 25, 50, 100 and 500 (the maximum document length). The average sentence length is 22 tokens. We see that the model is not simply relying on short range relationships, but is leveraging information about distant entity pairs, with accuracy increasing as the maximum distance considered increases. Note that all results are taken from the same model trained on the full unfiltered training set. Relation extraction is a heavily studied area in the NLP community. Most work focuses on news and web data (Doddington et al., 2004; Riedel et al., 2010; Hendrickx et al., 2009). 9 Recent neural net-9 And TAC KBP: https://tac.nist.gov 878 P R F1 Total Micro F1 44.8 50.2 47.3 Macro F1 34.0 29.8 31.7 Chemical/Disease marker/mechanism 46.2 57.9 51.3 therapeutic 55.7 67.1 60.8 Gene/Disease marker/mechanism 42.2 44.4 43.0 therapeutic 52.6 10.1 15.8 Chemical/Gene increases_expression 39.7 48.0 43.3 increases_MP 26.3 35.5 29.9 decreases_expression 34.4 32.9 33.4 increases_activity 24.5 24.7 24.4 aects_response 40.9 35.5 37.4 decreases_activity 30.8 19.4 23.5 aects_transport 28.7 23.8 25.8 increases_reaction 12.8 5.6 7.4 decreases_reaction 12.3 5.7 7.4 decreases_MP 28.9 7.0 11.0 Table 7: BRAN precision, recall and F1 results for the full CTD dataset by relation type. work approaches to relation extraction have focused on CNNs (dos Santos et al., 2015; Zeng et al., 2015) or LSTMs (Miwa and Bansal, 2016; Verga et al., 2016a; Zhou et al., 2016b) and replacing stage-wise information extraction pipelines with a single end-to-end model (Miwa and Bansal, 2016; Ammar et al., 2017; Li et al., 2017). These models all consider mention pairs separately. There is also a considerable body of work specifi-cally geared towards supervised biological relation extraction including protein-protein (Pyysalo et al., 2007; Poon et al., 2014; Mallory et al., 2015), drug-drug (Segura-Bedmar et al., 2013), and chemical-disease (Gurulingappa et al., 2012; Li et al., 2016a) interactions, and more complex events (Kim et al., 2008; Riedel et al., 2011). Our work focuses on modeling relations between chemicals, diseases, genes and proteins, where available annotation is often at the documentor abstract-level, rather than the chem_gene chem_disease gene_disease all Dataset 0.0 0.1 0.2 0.3 0.4 0.5 0.6 F 1 S c o r e 11 25 50 100 500 Figure 2: Performance on the CTD dataset when restricting candidate entity pairs by distance. Some previous work exists on cross-sentence relation extraction. Swampillai and Stevenson (2011) and Quirk and Poon (2017) consider featurized classifiers over cross-sentence syntactic parses. Most similar to our work is that of Peng et al. (2017), which uses a variant of an LSTM to encode document-level syntactic parse trees. Our work diers in three key ways. First, we operate over raw tokens negating the need for part-of-speech or syntactic parse features which can lead to cascading errors. We also use a feed-forward neural architecture which encodes long sequences far more eciently compared to the graph LSTM network of Peng et al. (2017). Finally, our model considers all mention pairs simultaneously rather than a single mention pair at a time. We employ a bi-ane function to form pairwise predictions between mentions. Such models have also been used for knowledge graph link prediction (Nickel et al., 2011; Li et al., 2016b), with variations such as restricting the bilinear relation matrix to be diagonal (Yang et al., 2015) or diagonal and complex (Trouillon et al., 2016). Our model is similar to recent approaches to graph-based dependency parsing, where bilinear parameters are used to score head-dependent compatibility (Kiperwasser and Goldberg, 2016; Dozat and Manning, 2017). We present a bi-ane relation attention network that simultaneously scores all mention pairs within a document. Our model performs well on three datasets, including two standard benchmark biological relation extraction datasets and a new, large and high-quality dataset introduced in this work. Our model out-performs the previous state of the art on the Biocreative V CDR dataset despite us-879 ing no additional linguistic resources or mention pair-specific features. Comparative toxicogenomics database: a knowledgebase and discovery tool for chemical genedisease networks. Nucleic acids research 37(suppl_1):D786D792. George Doddington, Alexis Mitchell, Mark Przy-bocki, Lance Ramshaw, Stephanie Strassel, and Ralph Weischedel. 2004. The automatic content extraction (ace) program tasks, data, and evaluation. In Proceedings of the Fourth International Conference on Language Resources and Evaluation . Timothy Dozat and Christopher D Manning. 2017. Deep biane attention for neural dependency parsing. 5th International Conference on Learning Representations . Philip Gage. 1994. A new algorithm for data compression. The C Users Journal 12(2):2338. Xavier Glorot, Antoine Bordes, and Yoshua Bengio. 2011. Deep sparse rectifier neural networks. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics . pages 315323. Jinghang Gu, Longhua Qian, and Guodong Zhou. 2016. Chemical-induced disease relation extraction with various linguistic features. Database 2016. Jinghang Gu, Fuqing Sun, Longhua Qian, and Guodong Zhou. 2017. Chemical-induced disease relation extraction via convolutional neural network. Database 2017. Our current model predicts only into a fixed schema of relations given by the data. However, this could be ameliorated by integrating our model into open relation extraction architectures such as Universal Schema (Riedel et al., 2013; Verga et al., 2016b). Our model also lends itself to other pairwise scoring tasks such as hypernym prediction, co-reference resolution, and entity resolution. We will investigate these directions in future work. We thank Ofer Shai and the Chan Zuckerberg Initiative / Meta data science team for helpful discussions. We also thank Timothy Dozat and Kyubyong Park for releasing their code.
abstain abstain abstain objective method method method result objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain objective abstain method result result objective objective method abstain abstain abstain abstain abstain method abstain method method abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method other abstain abstain method abstain abstain abstain abstain abstain method abstain method method abstain method method abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain other result abstain abstain abstain result result result other abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain result result abstain result result other abstain abstain abstain abstain abstain abstain abstain abstain result abstain other other other other other other method abstain abstain method abstain abstain method method method abstain method method objective method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain objective other other