Abstract Different Open Information Extraction (OIE) tasks require different types of information, so the OIE field requires strong adaptability of OIE algorithms to meet different task requirements. This paper discusses the adaptability problem in existing OIE systems and designs a new adaptable and efficient OIE system OIE@OIA as a solution. OIE@OIA follows the methodology of Open Information eXpression (OIX): parsing a sentence to an Open Information Annotation (OIA) Graph and then adapting the OIA graph to different OIE tasks with simple rules. As the core of our OIE@OIA system, we implement an end-to-end OIA generator by annotating a dataset (we make it open available) and designing an efficient learning algorithm for the complex OIA graph. We easily adapt the OIE@OIA system to accomplish three popular OIE tasks. The experimental show that our OIE@OIA achieves new SOTA performances on these tasks, showing the great adaptability of our OIE@OIA system. Furthermore, compared to other end-to-end OIE baselines that need millions of samples for training, our OIE@OIA needs much fewer training samples (12K), showing a significant advantage in terms of efficiency. Open Information Extraction (OIE) techniques are gradually attracting more and more attention (Christensen et al., 2011; Mausam et al., 2012; Corro and Gemulla, 2013; Angeli et al., 2015; Bhutani et al., 2016; Cui et al., 2018; Roy et al., 2019; Zhan and Zhao, 2020) as they build a bridge between language to knowledge. OIE tasks are generally designed for its information extraction requirements, which vary from verbal relations between entities (Banko et al., 2007; Etzioni et al., 2004), nominal attributes (Yahya et al., 2014; Pal and Mausam, 2016; Saha et al., 2017), and adverbial components (e.g., time) (Stanovsky et al., 2018). Even for the same type of information, the required facts may still differ. Table 1 shows the required form of facts of three popular OIE tasks: OIE2016 (Stanovsky and Dagan, 2016), Re-OIE2016 (Zhan and Zhao, 2020) and CaRB (Bhardwaj et al., 2019). The diversity of requirements is an essential feature in the field of OIE, which leads to the urgent need for OIE algorithms with strong adaptability. The adaptability problem in the OIE field has not been well addressed. There are two primary methodologies to obtain an OIE system: the rule-based approach and the end-to-end learning-based approach. The rule-based approaches (Christensen et al., 2011; Corro and Gemulla, 2013; Angeli et al., 2015; Bhutani et al., 2016; Gashteovski et al., 2017) use human-written or bootstrap-learned rules to convert linguistic structures of sentences into target facts. The end-to-end learning approaches (Stanovsky et al., 2018; Sun et al., 2018b,a; Roy et al., 2019; Ro et al., 2020; Kolluru et al., 2020; Liu et al., 2020) first build a dataset containing <sentence, facts> pairs and then use end-to-end learning to train a neural network as the OIE system. However, these methodologies develop a specific machine for every single task. When the requirements change, one must rewrite the complex rule system or re-annotate the data and then retrain the model. These methodologies fail to meet the need for strong adaptability in the OIE field. Recently, a concept called Open Information eXpression (OIX) was proposed by Sun et al. (2020) to address the adaptability issue of OIE algorithms. The idea of OIX is to introduce an intermediate layer between the language and OIE, which can express the sentence without information loss and be easily adapted to various OIE tasks. Sun et al. (2020) proposed a standard, called Open Information Annotation (OIA), to implement OIX. The OIA standard defines an annotation criterion of 6213 Fact OIE2016 Re-OIE2016 CaRB <[is], Ms. Lee, headmaster> (cid:55) (cid:51) (cid:51) <is responsible, Ms. Lee, for this> (cid:51) (cid:51) (cid:51) <told, Ms. Lee, Lily, she is responsible for this> (cid:51) (cid:55) (cid:51) <told, Ms. Lee, Jimmy, she is responsible for this> (cid:51) (cid:55) (cid:51) <told, Ms. Lee, Lily and Jimmy, she is responsible for this> (cid:55) (cid:51) (cid:55) Table 1: Facts defined in different OIE tasks, for the expression  Ms. Lee, the headmaster, told Lily and Jimmy she is responsible for this. natural language sentences, which aims to express all information of a sentence into a Predicate-Function-Argument structure, represented by a single-rooted DAG graph. In addition, they implemented a rule-based OIA system that generates OIA graphs from Universal Dependency graphs. Following the methodology of OIX/OIA, we implement an adaptable and efficient OIE system -OIE@OIA. The framework of OIE@OIA shown in Figure 1 has two components. The first one is the OIA generator, which converts a sentence into an OIA graph. We annotate a large OIA dataset (containing 12,543 training samples, 2,002 development samples, and 2,077 testing samples), develop an efficient learning algorithm to learn and inference the OIA graph, and finally build an end-to-end OIA graph learner. The second component is a group of adaptors, one for each OIE task. We show three popular OIE tasks focused on in this paper in the figure. Furthermore, one can write new adaptors for new OIE tasks, which are very simple, as shown in the following sections. The OIE@OIA system achieves the SOTA (or comparable) performance on three OIE tasks: OIE2016, Re-OIE2016, and CaRB, verifying the adaptability of our OIE@OIA system. Furthermore, our OIE@OIA only needs 12K samples to train, whereas existing end-to-end OIE methods typically need millions of training samples. This verifies the efficiency of our OIE@OIA system. An adaptable and efficient OIE system  OIE@OIA, achieving the SOTA performance on different OIE tasks. The first end-to-end OIA learning pipeline built on a large human-labeled OIA dataset (we make it open available) and an efficient algorithm for the OIA graph; Figure 1: The framework of OIE@OIA. In this section, we introduce the OIE@OIA framework, compare it with existing methodologies, and finally discuss its capability and limitation. The core of the OIE@OIA framework is an end-to-end learned OIA generator. To build the generator, we annotate a large dataset using sentences from English-EWT (version 2.4, containing 16K sentences ) 1 and design a neural-based algorithm for learning the OIA graph from sentences. The data annotation procedure and the learning procedure are detailed in Section 3. The standard OIA graph described in Sun et al. (2020) only defined three node types: Constant, Predicate, and Function. However, users may need more fine-grained type information about nodes, especially the type of predicates, to filter wanted facts. For instance, in building OIE systems based on OIA, we need to recognize verbal nodes, which act as the relationship descriptions of the OIE facts. In addition, in event logic graph construction (Ding et al., 2019), logical predicates are essential. With this consideration, we update the type field to a fine-grained version for nodes in the OIA graph according to its semantic function. The fine-grained node types are listed in Table 2. 1 https://lindat.mff.cuni.cz/ repository/xmlui/handle/11234/1-2988 6214 0 Ms. Lee ((0, 1),) Noun 1 the headmaster ((3, 4),) Noun appos 2 told (6,) Verbal pred.arg.1 3 Lee, the headmaster", "Lily and Jimmy", "she is responsible for this"> and <"is responsible", "she", "for this"> . VerbalPiP : In the OIA graph, for each verbal node with a prepositional child, we merge the child into the verbal node and apply the Verbal rule on the resultant OIA graph. This produces <"is responsible for", "she", "this"> for the sample in Figure 2 instead of <"is responsible", "she", "for this"> . Appos(be) : All edges like <A, appos, B> in OIA graphs are extracted to form the facts <be, A, B> . CoordSep : The fact tuples with coordination arguments are separated into multiple fact tuples, e.g., <told, ~, Lily and Jimmy, ~> is separated into <told, ~, Lily, ~> and <told, ~, Jimmy, ~> . Then, we implement the adaptors for the three tasks using the combinations of the above rules:  Adaptor@OIE 2016 = Verbal + CoordSep ;  Adaptor@Re-OIE 2016 = Verbal + Appos ([is]);  Adaptor@CaRB = VerbalPiP + Appos (is) + CoordSep . 2.3 Comparisons In this section, we compare OIE@OIA with existing OIE methodologies and show the difference in Table 3. The traditional rule-based OIE methods are generally based on a sentence annotation structure, such as dependency graphs or constituency graphs, and apply rules to convert the annotation structure into the OIE facts. This 6215 Methodology Rule-Based OIE End-to-End OIE OIE@OIA Sentence Annotation Dependency / Constituency OIA OIE Sensitiveness of Annotation No Yes Rule Complexity High Low Training Data 1 Million 12K Training Efficiency Low High Adaptation to New Task Rewrite Rules Relabel and Retrain New Adaptor Adaptation Cost May Be High May Be High Low Table 3: Comparisons among Rule-based OIE, End-to-End OIE , and OIE@OIA. pipeline is similar to our OIE@OIA, where OIE@OIA uses the OIA graph as the sentence annotation structure. However, the differences between the traditional annotation and OIA make the substantial differences between the rule-based OIE and our OIE@OIA. Since the traditional annotation dependency and constituency is not designed for the OIE task, one needs to write a complex rule system (or construct by bootstrapping) to convert those annotation structures into the OIE facts. However, for OIE@OIA, since the OIA is designed for OIE tasks, one can accomplish the conversion with straightforward rules, just like those described in the above section. As for the end-to-end OIE algorithms, existing methods are generally built on OpenIE4 dataset (Zhan and Zhao, 2020), which contains about 1 million training samples. However, differences may exist in the forms between the training dataset and the target task, so the performance may drop when adapting to new tasks. To limit the differences, one may need many new training samples and retrain the model. Our OIE@OIA can adapt to an extensive range of new tasks by introducing new adaptors, so it has much better adaptability. In addition, our experimental studies show that OIE@OIA needs only 12K samples for training to achieve new SOTA OIE performance, so it is much more efficient to implement an OIE system. 2.4 Capability and Limitations Besides the type of facts defined in the three popular tasks, one can extract more types of facts from OIA graphs. For example, since OIA graphs are naturally hierarchical, one can easily extract the nested facts, which can implement the target task of NestIE (Bhutani et al., 2016). One can also extract logical relationships between facts since OIA identifies the logical predicate nodes. We believe OIA can act as a general platform for various OIE tasks and provide better facts for downstream tasks based on OIE (Ding et al., 2019; Zhang et al., 2020). However, the current version of the OIE@OIA pipeline does not separate the compound noun phrase, making it unable to extract nominal relationships between different nominals within a compound noun phrase (Yahya et al., 2014). This is because current OIA graphs are phrase-level graphs and take noun phrases as single nodes. As an example,  the president of America " will form a single node in our OIA graph, and it is not able to identify the relationship between  the president " and  America " based on the graph. We left this problem as one of our future work. Converting a sentence into the OIA graph is the central operation of the OIE@OIA framework. We build an OIA dataset using active-learning-powered human labeling to implement such an operation. Then, we learn equivalent variants of the OIA graphs  Word-OIA graphs and convert them back to OIA-graphs, which overcomes the difficulty of learning the structures of the OIA graphs. We annotated sentences of English-EWT (version 2.4) for OIA. The annotation mainly follows the OIA graph definition given by Sun et al. (2020) but with some confusing or special cases being clarified. In addition, we introduce more detailed type information for the node. The obtained dataset contains 12,543 training samples, 2,002 development samples, and 2,077 testing samples. Each sample is a sentence-graph pair. On average, a graph has 7.74 nodes and 6.95 edges, and a node comprises 1.98 words. We make the updated 6216 annotation standard and dataset open available 3 . Auxiliary Annotation System. To improve data annotation efficiency, we generate an initial OIA graph for each input sentence using the existing rule-based OIA system (Sun et al., 2020). For node types initialization, we align the phrases with the POS tags in English-EWT v2.4 and assign heuristic types based on the POS tags of the head words. Then we develop an annotation tool for the annotator to modify the adapted graphs with ease. Active Learning. The samples in the dev set and test set are all human-labeled. For the training set, we first randomly labeled 2,000 samples, then trained a model using the proposed learning method (described in  3.4) and started the active-learning procedure. The data labeling order of unlabeled samples was determined by the difference between the rule-generated results and the predicted results. We labeled 200 samples in each active learning iteration and stopped the iteration when the performance of the trained did not improve on the dev set. As a result, we manually annotated about 74% of the training data and treated the rule-generated results as the true labels of the rest 26% training data. Quality Control. The data annotation was done by three postgraduate/doctoral students of linguistics. Two annotators first label each sample. If there is a disagreement, the third annotator will be involved for discussion and voting. The initial agreement ratio between the two annotators is about 80%, and the final agreement ratio after the discussion (no vote needed) is higher than 93%. The annotation of the rest 7% data is obtained by voting. The node of OIA graphs consists of a sequence of symbols, placeholders, and words. We call such a graph Generalized Phrase Graph ( GGPG = ( P , S ) , where P is the set of generalized nodes and S is the edge set). Directly learning the graph is difficult due to the very large decision space caused by the complexity of OIA nodes. The decision space is large even in the simplest situation that each node consists of consecutive words (a span in the sentence without any symbol or placeholder outside the sentence). Since the target number of nodes is unknown, the number of candidate node 3 https://github.com/sunbelbd/ Open-Information-eXpression sets to be considered is exponential to the number of words in the sentence. Due to this large decision space, it is very difficult to learn a good candidate set of nodes for OIA graphs as the first step task in a stage-wise approach. If one prefers the end-to-end approach to reduce the error propagation between tasks in each stage by jointly leaning the nodes and edges, the decision space will be even much larger. Fortunately, we have the following proposition connecting the Generalized Phrase Graph with Word Graph , where Word Graph is a graph with each graph node corresponding to one and only one word of the sentence: Proposition. For any Generalized Phrase Graph GGPG = ( P , S ) , there is a one-to-one corresponding Word Graph GW = ( W , S (cid:48) ) in the sense that GGPG and GW can convert to each other without loss of information, where the labels of S (cid:48) are independent to word nodes W . Proof. We split each generalized phrase node into a chain of nodes of symbols, placeholders, or words, connected in order with the edge next , and connect all the edges from parents/children to the first node of the chain. In this new graph, each word is a single node. We replace each type of path between two words (or the virtual root) containing only symbols/placeholders into a single edge with a correspondingly designed edge label and remove the original path. The resulting graph is a valid word graph. A constructive procedure to convert a General Phrase Graph into the Word Graph is shown in Appendix A. The OIA graphs are special cases of GPG, and the properties of the OIA graphs can make the procedure much simpler. We discuss these details in Appendix B. With the above procedure, we can convert a complex OIA graph into an equivalent simple Word-OIA graph (as shown in Figure 3), which is a single-rooted DAG where each node is a word in the original sentence. Each node in the Word-OIA graph has one category attribute type (by sharing the type of OIA node it belongs to) and two boolean attributes arg_whether and missing_be (described in Appendix B). (a) The OIA graph for the sentence A. not missing_be=True sure next_word dependents upper_parataxis do parataxis It pred.arg.1 n't next_word I pred.arg.1 know next_word of the Word OIA graph. The node number of the Word-OIA graph is fixed N , so we only need to learn the possible N ( N  1) edges, and thus the learning complexity is significantly reduced. The structure of the Word-OIA graph is similar to that of the dependency graph so that the semantic dependency graph learning procedure can be applied to the learning of the Word-OIA graph. We build our learning procedure based on pretrained BERT models (Devlin et al., 2019). Given a sentence S = [ w 1 ,    , w N ] , we generate the representation r i of each word by: R = BERT ( S ) , where R = [ r 1 ,    , r N ] . Then we learn the properties of nodes and the graph structures using these representations. Node Attribute Learning. For node attribute prediction, we build a one-layer MLP classifier above r i to learn each attribute a k for word w i : p ( node ) ki = p ( a k | w i ) = Softmax ( MLP ( node ) k ( r i )) . The loss of node attribute prediction is defined as: L node = 1 N (cid:88) k (cid:88) i (cid:96) CE (cid:16) p ( node ) ki , y ( node ) ki (cid:17) , where y ( node ) ki denotes the target attribute value of the corresponding node of w i and (cid:96) CE denotes the cross-entropy loss function. Edge Learning. Following the protocol of Dozat and Manning (2018), we divide the structure learning into two steps: given two nodes, we firstly determine if there is an edge between them; if so, we then determine the type of the edge. It avoids introducing an empty edge type in the second step, which will overwhelm the other edge types. For the two-step learning, we use the biaffine-based graph learning approach (Dozat and Manning, 2017, 2018). In the first step, for two words w i and w j , we learn the representations of each word as the start and end node of an edge: h ( es ) i = MLP ( es ) ( r i ) , h ( ee ) i = MLP ( ee ) ( r i ) . being an edge e ij between w i and w j is: s ( edge ) ij = Bia ( edge ) ( h ( es ) i , h ( ee ) j ) , p ( edge ) ij =  (cid:16) MLP ( edge ) ( s ( edge ) ij ) (cid:17) . Here, the i -th dimension of Bia ( x 1 , x 2 ) is: x (cid:62) 1 U i x 2 + w (cid:62) i ( x 1  x 2 ) + b i , where U i , w i , b i denotes a trainable matrix, vector, and scalar, respectively. The corresponding graph topology loss on S is defined as follows: L topo = 1 N 2 (cid:88) ij (cid:96) CE (cid:16) p ( edge ) ij , y ( edge ) ij (cid:17) . Then we learn the label of each edge. We learn the representations of start-node and end-node of an edge by: h ( ls ) i = MLP ( ls ) ( r i ) , h ( le ) j = MLP ( le ) ( r j ) . The probability of the label l ij is: s ( label ) ij = Bia ( label ) ( h ( ls ) i , h ( le ) j ) , p ( label ) ij = Softmax (cid:16) MLP ( label ) ( s ( label ) ij ) (cid:17) , and the edge prediction loss on S is defined as: L label = (cid:80) ij I ( y ( edge ) ij = 1) (cid:96) CE (cid:16) p ( label ) ij , y ( label ) ij (cid:17) (cid:80) ij I ( y ( edge ) ij = 1) , where I (  ) is the indicator function. is the concatenation operator. Multi-Task Learning. We learn the Word-OIA graph in a multi-task style, that is, optimize a linear combination of the losses: L =  L topo +  L label + (1     ) L node . In the inference phase, we accomplish the following steps to generate the Word-OIA graph: Node Attribute Prediction. For each node w i , its type t i is predicted by:  y ki = arg max p ( node ) ki . However, because the label prediction may be incorrect, constructing the graph using the above predictions may result in an invalid graph (dis-connected graph, edge conflicted, etc.). So in practice, we develop a greedy search strategy to construct the graph step-by-step while maintaining the validness of the graph all the time. First, we select the edge with the highest value of p ( label ) ij for all edges with p ( edge ) ij > 0 . 5 . Then, we identify conflicted edges with the selected edges and set their corresponding values in p ( label ) ij to zero. The above process iterates several times until all edge types are set. In addition, the resulting graph may consist of several disconnected sub-graphs. In this case, we iteratively select the edge with the highest p ( label ) ij to connect it to the sub-graph to which the predicted root belongs. It guarantees that the generated Word-OIA graph is valid. For a predicted Word-OIA graph, we reverse the OIA to the Word-OIA procedure to obtain the OIA graph. Specifically, we first collect nodes in Word-OIA graph chained by next_word and related arcs ( prev_arg , pos_arg ) to form the nodes in OIA graph. Then we identify the special structure such as edge upper_parataxis and add special node like Parataxis and Missing to OIA graph. We add (be) to the node span if missing_be is true. The Whether node is added as the parent of current node if arg_whether is true. Last, we connect the nodes using the learned arc labels in Word-OIA. The type of the phrase node in the OIA graph is set as the majority type of its constituted words in the corresponding Word-OIA graph. The experiment is conducted on the PaddlePaddle deep learning platform 4 , and the pre-trained BERT model is provided by the PaddleNLP project 5 . Following Che et al. (2020), the hidden size of MLP ( edge ) and MLP ( label ) is set to 500 and 100, respectively. The hidden sizes of MLPs used in node attribute predictions are set to 500. The model is trained with the classifier's dropout rate being set to 0.1 and Adam optimizer with a learning rate of 10  5 . and  in loss function are searched using 4 https://www.paddlepaddle.org.cn 5 https://github.com/PaddlePaddle/ PaddleNLP 6219 Level Metric Performance Node type Acc 0.951 arg_whether Acc 0.998 missing_be Acc 0.998 Edge P/R 0.847 / 0.851 Graph Acc 0.528 Table 4: Performance of Word-OIA prediction, where P/R means Precision/Recall, Acc means Accuracy. Evaluation Metric. In this experiment, the evaluation metrics for all measurements are based on exact match, that is, score 1 if exactly the same, otherwise 0. For nodes, we compare their node expressions. For a Word-OIA node, the node expression is the word index in the sentence; for an OIA node, the node expression is the phrase based on the word indexes it contains. For edges, we compare the triplets of <start_node_expression, edge_label, end_node_expression> . For graphs, we test whether the two graphs' node sets and edge sets are exactly the same. The node precision and recall of Word-OIA are always 1.0 since the nodes correspond to the words in the sentence. The performance of node attribute prediction is illustrated by the upper part of Table 4. The precision/recall of edge prediction is shown in the middle part of Table 4. We also report the accuracy of graph structure in the last part of Table 4. 4.2 Performance on OIA We report the performances of the nodes, edges, and the whole graph structure of the recovered OIA graphs in the lower half of the Table 5. Note that the graph-structure accuracy of the OIA graph is slightly lower than that of the Word-OIA graph. It is because the graph structure of the OIA graph is related to the node attributes arg_whether and missing_be . Since there are tiny proportions of bad cases in predicting these two attributes, the graph-structure accuracy of the OIA graph is lower. As a comparison, we report the performances of the rule baseline (Sun et al., 2020) in upper part of Table 5. We can see that the proposed method achieves significant improvement over the rule baseline, e.g., improving the graph structure Generator Level Metric Performance Rule Node P/R 0.796 / 0.855 Rule Edge P/R 0.530 / 0.585 Rule Graph Acc 0.373 Neural Node P/R 0.893 / 0.877 Neural Edge P/R 0.709 / 0.688 Neural Graph Acc 0.525 Table 5: Performance of OIA graph prediction. accuracy by 15.2%. We believe the learning-based approach solves several problems in the rule-based approach: 1) limitation of expressiveness of Universal Dependency, 2) mistakes in Universal Dependency and Enhanced++ (Schuster and Manning, 2016) parsers, and 3) failure of rules to cover the complex combination of situations. We also evaluate the accuracy of the node type of the recovered OIA graphs. Among the 89.3% correctly identified nodes, 96.4% of them are labeled with the correct node types by the voting of nodes in the Word-OIA graphs. We reviewed the error cases to find the limitations in the graph generation process. We find several common issues that lead to incorrect graphs. Long Tail Words and Edges. About 33% errors are caused by the long tail words and edge labels. The out-of-vocabulary words lead to problematical word representations. Rarely used edge labels (e.g., discourse) tend to be predicted as other frequent edge labels. Granularity Issue. The granularity or boundary of the node may be controversial in prediction results. For example, the phrase turn out to be' can be a predicate, but it also makes sense that turn out' and to be' form a nested relation. Such granularity issues cause about 25% errors in both predicate node and constant node. Mining idioms can further clarify the boundary of expression with refined strategy. This belongs to our future work. Ambiguous Modification. A prepositional phrase can be used to modify either a noun or a verb in its context. This ambiguity leads to about 17% of graph-level errors. For example, in sentence I love all the roles in this play , prepositional phrase in this play is the modifier of all the roles . Thus, they should be in the same noun node of the ground-truth OIA graph. However, in the predicted graph, 6220 Systems OIE2016 Re-OIE2016 CaRB AUC F1 AUC F1 AUC F1 R u l e B a s e d Stanford (Angeli et al., 2015) 7.9 13.6 11.5 16.7 13.4 23.0 OLLIE (Mausam et al., 2012) 20.2 38.6 31.3 49.5 22.4 41.1 NestIE (Bhutani et al., 2016) 37.7 43.8 32.1 42.2 19.4 31.1 PropS (Stanovsky and Dagan, 2016) 32.0 54.4 43.3 64.2 12.6 31.9 MinIE (Gashteovski et al., 2017) 35.0 41.0 45.5 47.8 28.1 41.3 ClausIE (Corro and Gemulla, 2013) 36.4 58.0 46.4 64.2 22.4 44.9 OIE@RuleOIA 37.3 54.6 63.3 75.0 32.4 45.6 L e a r n i n g B a s e d OpenIE4 (Christensen et al., 2011) 40.8 58.8 50.9 68.3 27.2 48.8 BIO (Zhan and Zhao, 2020) 46.2 68.6 71.9 80.3 27.7 46.6 SpanOIE (Zhan and Zhao, 2020) 48.9 68.7 65.8 77.0 30.0 49.4 BiLSTM + BERT (Ro et al., 2020) -72.1 81.3 30.6 50.6 Multi2OIE (BERT) (Ro et al., 2020) -74.6 83.9 32.6 52.3 OIE@OIA (BERT) 54.3 71.6 76.9 85.3 33.9 51.1 Table 6: OIE performance on OIE2016, Re-OIE2016 and CaRB. in this play may become the sub-tree of verb love . It is a common error in parsing tasks that a model may incorrectly choose the headword for a modifier. We believe better language modeling will ameliorate this problem. We further evaluate the applicability of OIA as an intermediate layer between language and OIE, i.e., OIE@OIA, on three tasks: OIE2016, Re-OIE2016, CaRB. We compared OIE@OIA with six rule-based systems and five learning-based systems. Evaluation Metric. The performances of baseline systems on OIE2016 are from Zhan and Zhao (2020) while that on Re-OIE2016 and CaRB are from Ro et al. (2020), except that NestIE (Bhutani et al., 2016) is implemented using the code from the author. We evaluate the extraction results of the proposed methods, OIE based on rule OIA and NestIE with metric AUC and optimal F1 following the setting of the released codes 6 . The performance of our OIE@OIA system is shown in Table 6. We observe that OIE@OIA achieves better performance than most existing baselines, including learning-based methods trained on millions of samples. This result justifies the effectiveness of OIE@OIA for OIE. We think the reason for this phenomenon is three-fold: Firstly , compared with the annotation of OIA, the annotation of a single OIE task is 6 www.github.com/zhanjunlang/Span_OIE , and www.github.com/youngbin-ro/Multi2OIE sparse. Given a sentence, it will only annotate phrases with interesting relationships. For the other phrases and relationships, it will not annotate. In contrast, in OIA, all the phrases and relationships will be annotated. So based on a single sample, the annotation in OIA is much more informative than that in any single OIE task. Secondly , if treating the recognition of a type of facts as a task, learning of OIA can be seen as a multi-task learning scenario, and different tasks can augment each other. Thirdly , OIA is designed as an intermediate layer between language and OIE. Thus, during the standard design of OIA, it has considered the compatibility between OIA and OIE, which makes it easier to adapt OIA to different OIE tasks. We introduce an adaptable and efficient Open Information Extract system called OIE@OIA. To implement OIE@OIA, we annotate and release an OIA dataset containing about 16K sentences, design an efficient learning algorithm, and build an easy-to-implement rule system to adapt OIA graphs to different OIE tasks. Empirical studies on three popular OIE tasks show that our OIE@OIA system can achieve new SOTA performances on these tasks, using only 12K training sentences. It verifies the great advantage of our system in both effectiveness and efficiency over the previous learning-based baselines, which usually require millions of training samples to achieve comparable performance.
abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain other abstain abstain method abstain abstain objective abstain result abstain result method abstain objective objective method abstain method abstain abstain method abstain method other other abstain abstain method method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain method method method abstain method abstain abstain method abstain method abstain abstain abstain abstain abstain abstain other abstain abstain method method abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain method method abstain method abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain result abstain abstain other abstain abstain abstain abstain result result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method objective result