Distance-based knowledge graph embeddings have shown substantial improvement on the knowledge graph link prediction task, from TransE to the latest state-of-the-art RotatE . However, complex relations such as N-to-1, 1-to-N and N-to-N still remain challenging to predict. In this work, we propose a novel distance-based approach for knowledge graph link prediction. First we extend the RotatE from 2D complex domain to high dimensional space with orthogonal transforms to model relations. The orthogonal transform embedding for relations keeps the capability for modeling symmetric/anti-symmetric, inverse and compositional relations while achieves better modeling capacity. Second, the graph context is integrated into distance scoring functions directly. Specifically, graph context is explicitly modeled via two directed context representations. Each node embedding in knowledge graph is augmented with two context representations, which are computed from the neighboring outgoing and incoming nodes/edges respectively. The proposed approach improves prediction accuracy on the difficult N-to-1, 1-to-N and N-to-N cases. Our experimental results show that it achieves state-of-the-art results on two common benchmarks FB15k-237 and WNRR-18, especially on FB15k-237 which has many high in-degree nodes. Code available at https://github. com/JD-AI-Research-Silicon-Valley/ KGEmbedding-OTE . Knowledge graph is a multi-relational graph whose nodes represent entities and edges denote relationships between entities. Knowledge graphs store facts about people, places and world from various sources. Those facts are kept as triples (head entity, relation, tail entity) and denoted as ( h, r, t ) . A large number of knowledge graphs, such as Freebase (Bollacker et al., 2008), DBpedia (Auer et al., 2007), NELL (Carlson et al., 2010) and YAGO3 (Mahdisoltani et al., 2013), have been built over the years and successfully applied to many domains such as recommendation and question answering (Bordes et al., 2014; Zhang et al., 2016). However, these knowledge graphs need to be updated with new facts periodically. Therefore many knowledge graph embedding methods have been proposed for link prediction that is used for knowledge graph completion. Knowledge graph embedding represents entities and relations in continuous vector spaces. Started from a simple and effective approach called TransE (Bordes et al., 2013), many knowledge graph embedding methods have been proposed, such as TransH (Wang et al., 2014), DistMult (Yang et al., 2014), ConvE (Dettmers et al., 2018) to the latest RotatE (Sun et al., 2019) and QuatE (Zhang et al., 2019). Though much progress has been made, 1-to-N, N-to-1, and N-to-N relation predictions (Bor-des et al., 2013; Wang et al., 2014) still remain challenging. In Figure 1, relation profession demonstrates an N-to-N example and the corresponding edges are highlighted as green. Assuming the triple (SergeiRachmaninoff, Profession, Pianist) is unknown. The link prediction model takes SergeiRachmaninoff and relation Profession and rank all entities in the knowledge graph to predict Pianist. Entity SergeiRachmaninoff connected to multiple entities as head entity via relation profession, while Pianist as a tail entity also reaches to multiple entities through relation profession. It makes the N-to-N prediction hard because the mapping from certain entity-relation pair could lead to multiple different entities. Same issue happens with the case of 1-to-N and N-to-1 predictions. The recently proposed RotatE (Sun et al., 2019) Figure 1: Snapshot of knowledge graph in FB15k-237. In this work, a novel distance-based knowledge graph embedding called orthogonal transform embedding ( OTE ) with graph context is proposed to alleviate the 1-to-N, N-to-1 and N-to-N issues, while keeps the desired relation patterns as RotatE . First, we employ orthogonal transforms to represent relations in high dimensional space for better modeling capability. The Orthogonal transform embedding also models the symmetry/antisymmery, inversion and compositional relation patterns just as RotatE does. RotatE can be viewed as an orthogonal transform in 2D complex space. Second, we integrate graph context directly into the distance scoring, which is helpful to predict 1-to-N, N-to-1 and N-to-N relations. For example, from the incomplete knowledge graph, people find useful context information, such as (SergeiRach-maninoff, role, Piano) and (SergeiRachmaninoff, Profession, Composer) in Figure 1. In this work, each node embedding in knowledge graph is augmented with two graph context representations, computed from the neighboring outgoing and incoming nodes respectively. Each context representation is computed based on the embeddings of the neighbouring nodes and the corresponding relations connecting to these neighbouring nodes. These context representations are used as part of the distance scoring function to measure the plausibility of the triples during training and inference. We show that OTE together with graph context modeling performs consistently better than RotatE on the standard benchmark FB15k-237 and WN18RR datasets. A new orthogonal transform embedding OTE , is proposed to extend RotatE from 2D space to high dimensional space, which also models sym-metry/antisymmery, inversion and compositional relation patterns; A directed graph context modeling method is proposed to integrate knowledge graph context (including both neighboring entity nodes and relation edges) into the distance scoring function; Experimental results of OTE on standard benchmark FB15k-237 and WN18RR datasets show consistent improvements over RotatE , the state of art distance-based embedding model, especially on FB15k-237 with many high in-degree nodes. On WN18RR our results achieve the new state-of-the-art performance. Knowledge graph embedding could be roughly categorized into two classes (Wang et al., 2017): distance-based models and semantic matching models. Distance-based model is also known as additive models, since it projects head and tail entities into the same embedding space and the distance scoring between two entity embeddings is used to measure the plausibility of the given triple. TransE (Bordes et al., 2013) is the first and most representative translational distance model. A series of work is conducted along this line such as TransH (Wang et al., 2014), TransR (Lin et al., 2015) and TransD (Ji et al., 2015) etc. RotatE (Sun et al., 2019) further extends the computation into complex domain and is currently the state-of-art in this category. On the other hand, Semantic matching models usually take multiplicative score functions to compute the plausibility of the given triple, such as DistMult (Yang et al., 2014), ComplEx (Trouillon et al., 2016), ConvE (Dettmers et al., 2018), TuckER (Balazevic et al., 2019) and QuatE (Zhang et al., 2019). ConvKB (Nguyen et al., 2017) and CapsE (Nguyen et al., 2019) further took the triple as a whole, and fed head, relation and tail embeddings into convolutional models or capsule networks. The above knowledge graph embedding methods focused on modeling individual triples. However, they ignored knowledge graph structure and did not take advantage of context from neighbouring nodes and edges. This issue inspired the usage of graph neural networks (Kipf and Welling, 2016; Velickovic et al., 2017) for graph context modeling. Encoder-decoder framework was adopted in (Schlichtkrull et al., 2017; Shang et al., 2019; Bansal et al., 2019). The knowledge graph structure is first encoded via graph neural networks and the output with rich structure information is passed to the following graph embedding model for prediction. The graph model and the scoring model could be end-to-end trained together, or the graph encoder output was only used to initialize the entity embedding (Nathani et al., 2019). We take another approach in this paper: we integrate the graph context directly into the distance scoring function. Orthogonal transform is considered to be more stable and efficient for neural networks (Saxe et al., 2013; Vorontsov et al., 2017). However, to optimize a linear transform with orthogonal property reserved is not straightforward. Soft constraints could be enforced during optimization to encourage the learnt linear transform close to be orthogonal. Bansal et al. (2018) extensively compared different orthogonal regularizations and find regularizations make the training faster and more stable in different tasks. On the other hand, some work has been done to achieve strict orthogonal during optimization by applying special gradient update scheme. Harandi and Fernando (2016) proposed a Stiefel layer to guarantee fully connected layers to be orthogonal by using Reimannian gradients. Huang et al. (2017) consider the estimation of orthogonal matrix as an optimization over multiple dependent stiefel manifolds problem and solve it via eigenvalue decomposition on a proxy parameter matrix. Vorontsov et al. (2017) applied hard constraint on orthogonal transform update via Cayley transform. In this work, we construct the orthogonal matrix via Gram Schmidt process and the gradient is calculated automatically through autograd mechanism in PyTorch (Paszke et al., 2017). We consider knowledge graph as a collection of triples D = {( h, r, t )} with V as the graph node set, and R as the graph edge set. Each triple has a head entity h and tail entity t , where h, t  V . Relation r  R connects two entities with direction from head to tail. As discussed in the introduction section, 1-to-N, N-to-1 and N-to-N relation prediction (Bordes et al., 2013; Wang et al., 2014) are difficult to deal with. They are addressed in our proposed approach by: 1) orthogonal relation transforms that operate on groups of embedding space. Each group is modeled and scored independently, and the final score is the sum of all group scores. Hence, each group could address different aspects of entity-relation pair and alleviate the 1-to-N and N-to-N relation mapping issues; and 2) directed graph context to integrate knowledge graph structure information to reduce the ambiguity. Next, we first briefly review RotatE that motivates our orthogonal transform embedding ( OTE ), and then describe the proposed method in details. OTE is inspired by RotatE (Sun et al., 2019). In RotatE , the distance scoring is done via Hadamard production (element-wise) defined on the complex domain. Given a triple ( h, r, t ) , the corresponding embedding are e h ,  r , e t , where e h and e t  R 2 d ,  r  R d , and d is the embedding dimension. For each dimension i , e [ 2 i ] and e [ 2 i + 1 ] are corresponding real and imaginary components. The projection  e t of t from corresponding relation and head entities is conducted as an orthogonal transform as below: [  e t [ 2 i ]  e t [ 2 i + 1 ]] = M r ( i ) [ e h [ 2 i ] e h [ 2 i + 1 ]] = [ cos  r ( i )  sin  r ( i ) sin  r ( i ) cos  r ( i ) ] [ e h [ 2 i ] e h [ 2 i + 1 ]] where M r ( i ) is a 2D orthogonal matrix derived from  r . Though RotatE is simple and effective for knowledge graph link prediction, it is defined in 2 D complex domain and thus has limited modeling capability. A natural extension is to apply similar operation on a higher dimensional space. We use e h , M r , e t to represent embeddings of head, relation and tail entity, where e h , e t  R d , and d is the dimension of the entity embedding. The entity embedding e x , where x = { h, t } , is further divided into K sub-embeddings, e.g., e x = [ e x ( 1 ) ;  ; e x ( K )] , where e x ( i )  R d s and d = K  d s . M r is a collection of K linear transform matrix M r = { M r ( 1 ) ,  , M r ( K )} , and M r ( i )  R d s  d s . For each sub-embedding e t ( i ) of tail t , we define the projection from h and r to t as below:  e t ( i ) = f i ( h, r ) =  ( M r ( i )) e h ( i ) (1) where  is the Gram Schmidt process (see details in Section 3.3) applied to square matrix M r ( i ) . 1 is re-written as  e t ( i ) = diag ( exp ( s r ( i )))  ( M r ( i )) e h ( i ) (2) Then, the corresponding distance scoring function is defined as d (( h, r ) , t ) = K  i = 1 (  e t ( i )  e t ( i )) (3) For each sub-embedding e h ( i ) of head h , we define the projection from r and t to h as below:  e h ( i ) = diag ( exp (  s r ( i )))  ( M r ( i )) T e t ( i ) (4) where the reverse project from tail to head is simply transposing the  ( M r ( i )) and reversing the sign of s r . The output transform  ( M r ( i )) is an orthogonal matrix derived from M r ( i ) . e t is the concatenation of all sub-vector  e t ( i ) from Eq. 1, e.g.,  e t = f ( h, r ) = [  e t ( 1 ) ;  ;  e t ( K )] . The L 2 norm of e h ( i ) is preserved after the orthogonal transform. We further use a scalar tensor s r ( i )  R d s to scale the L 2 norm of each group of embedding separately. Eq. Then, the corresponding distance scoring function is defined as d ( h, ( r, t )) = K  i = 1 (  e h ( i )  e h ( i )) . We employ Gram-Schmidt process to orthogonalize a linear transform into an orthogonal transform (i.e.,  ( M r ( i )) in Section 3.2). The Gram-Schmidt process takes a set of tensor S = { v 1 ,  , v k } for k  d s and generates an orthogonal set S  = { u 1 ,  , u k } that spans the same k  dimensional subspace of R d s as S . t i = v k  k  1  j = 1  v k , t j   t j , t j  t j (6) u i = t i  t i  (7) where t 1 = v 1 ,  t  is the L 2 norm of vector t and  v, t  denotes the inner product of v and t . Orthogonal transform has many desired properties, for example, the inverse matrix is obtained by simply transposing itself. It also preserves the L 2 norm of a vector after the transform. For our work, we are just interested in its property to obtain inverse matrix by simple transposing. This saves the number of model parameters (see Table 3). It can be easily proved that OTE has the ability to model and infer all three types of relation patterns: symmetry/antisymmetry, inversion, and composition as RotatE does. The proof is listed in Appendix A. It should be noted that, M r ( i ) is calculated every time in the neural networks forward computation to get orthogonal matrix  ( M r ( i )) , while the corresponding gradient is calculated and propagated back to M r ( i ) via autograd computation within PyTorch during the backward computation. It eliminates the need of special gradient update schemes employed in previous hard constraint based orthogonal transform estimations (Harandi and Fernando, 2016; Vorontsov et al., 2017). In our experiments, we initialize M r ( i ) to make sure they are with full rank 1 . During training, we also keep checking the determinant of M r ( i ) . We find the update is fairly 1 A real random matrix has full rank with probability 1 (Slinko, 2000). We use different random seeds to make sure the generated matrix is full rank. stable that we don't observe any issues with sub-embedding dimensions varied from 5 to 100 . The knowledge graph is a directed graph: valid triple ( h, r, t ) does not mean ( t, r, h ) is also valid. Therefore, for a given entity in knowledge graph, there are two kinds of context information: nodes that come into it and nodes that go out of it. Specially, in our paper, for each entity e , we consider the following two context settings: 1. If e is a tail, all the (head, relation) pairs in the training triples whose tail is e are defined as Head Relation Pair Context . 2. If e is a head, all the (relation, tail) pairs in the training triples whose head is e are defined as Relation Tail Pair Context . Figure 1 demonstrates the computation of graph context for a testing triple (SergeiRachmaninoff, profession, Pianist). Edges for relation profession are colored as green. Entities marked with  are head entities to entity Pianist, and these entities and corresponding relations to connect Pianist form the head relation pair context of Pianist. While entities with  are tail entities for entity SergeiRachmaninoff. Those entities and corresponding relations are the relation tail graph context of entity SergeiRachmaninoff. For a given tail t , all head-relation pairs ( h  , r  ) of the triples with tail as t are considered as its graph context and denoted as Ng ( t ) . First, we compute the head-relation context representation  e ct as the average from all these pairs in Ng ( t ) as below:  e ct =  ( h  ,r  )  Ng ( t ) f ( h  , r  ) + e t  Ng ( t ) + 1 (8) where e t is the embedding of the tail t , f ( h  , r  ) is the representation of ( h  , r  ) induced from Eq. 2. We use e t in Eq. 8 to make the computation of context representation possible when Ng ( t ) is empty. This can be viewed as a kind of additive smoothing for context representation computation. Then, we compute the distance of the head-relation context of t and the corresponding orthogonal transform based representation of a triple ( h, r, t ) as follow. where  e t ( i ) is computed from Eq. 2. There is no new parameter introduced for the graph context modeling, since the message passing is done via OTE entity-relation project f ( h  , r  ) . The graph context can be easily applied to other translational embedding algorithms, such as RotatE and TransE etc, by replacing OTE . For a given head h , all relation-tail pairs ( r  , t  ) of the triples with head as h are considered as its graph context and denoted as Ng ( h ) . First, we compute the relation-tail context representation  e ch as the average from all these pairs in Ng ( h ) as below:  e ch =  ( r  ,t  )  Ng ( h ) f ( r  , t  ) + e h  Ng ( h ) + 1 (10) where f ( r  , t  ) is computed from Eq. 4. Then, we compute the distance of the relation-tail context of h and the corresponding orthogonal transform based representation of a triple ( h, r, t ) as follow. We further combine all four distance scores (Eq. 3, Eq. 5, Eq. 9 and Eq. 11) discussed above as the final distance score of the graph contextual orthogonal transform embedding ( GC-OTE ) for training and inference Therefore the full GC-OTE model can be seen as an ensemble of K local GC-OTE models. This view provides an intuitive explanation for the success of GC-OTE. Optimization Self-adversarial negative sampling loss (Sun et al., 2019) is used to optimize the embedding in this work, L =   p ( h  , r, t  ) log  ( d all ( h  , r, t  )   )  log  (   d all ( h, r, t )) (13) where  is a fixed margin,  is sigmoid function, ( h  , r, t  ) is negative triple, and p ( h  , r, t  ) is the negative sampling weight defined in (Sun et al., 2019). Two commonly used benchmark datasets (FB15k-237 and WN18RR) are employed in this study to evaluate the performance of link prediction. FB15k-237 (Toutanova and Chen, 2015) dataset contains knowledge base relation triples and textual mentions of Freebase entity pairs. The knowledge base triples are a subset of the FB15K (Bordes et al., 2013), originally derived from Freebase. The inverse relations are removed in FB15k-237. WN18RR (Dettmers et al., 2018) is derived from WN18 (Bordes et al., 2013), which is a subset of WordNet. WN18 consists of 18 relations and 40,943 entities. However, many text triples obtained by inverting triples from the training set. Thus WN18RR (Dettmers et al., 2018) is created to ensure that the evaluation dataset does not have test leakage due to redundant inverse relation. Each dataset is split into three sets for: training, validation and testing, which is same with the setting of (Sun et al., 2019). The statistics of two data sets are summarized at Table 1. Only triples in the training set are used to compute graph context. Following the evaluation protocol in (Dettmers et al., 2018; Sun et al., 2019), each test triple ( h, r, t ) is measured under two scenarios: head focused ( ? , r, t ) and tail focused ( h, r, ? ) . For each case, the test triple is ranked among all triples with masked entity replaced by entities in knowledge graph. Those true triples observed in either train/validation/test set except the test triple will be excluded during evaluation. Top 1, 3, 10 (Hits@1, Hits@3 and Hits@10), and the Mean Reciprocal Rank (MRR) are reported in the experiments. Hyper-parameter settings The hyper-parameters of our model are tuned by grid search during training process, including learning rate, embedding dimension d and sub-embedding dimension d s . In our setting, the embedding dimension is defined as the number of parameters in each entity embedding. Each entity embedding consists of K sub-embeddings with dimension d s , i.e., d = K  d s . There are two steps in our model training: 1) the model is trained with OTE or RotatE models, and 2) graph context based models are fine tuned on these pre-trained models. The parameter settings are selected by the highest MRR with early stopping on the validation set. We use the adaptive moment (Adam) algorithm (Kingma and Ba, 2014) to train the models. Specially, for FB15k-237, we set embedding dimension d = 400 , sub-embedding dimension d s = 20 , and the learning rates to 2 e 3 and 2 e 4 for pre-training and fine-tuning stages respectively; for WN18RR dataset, we set d = 400 , d s = 4 , and the learning rates to 1 e 4 and 3 e 5 for pre-training and fine-tuning stages. Implementation Our models are implemented by PyTorch and run on NVIDIA Tesla P40 Graphics Processing Units. The pre-training OTE takes 5 hours with 240,000 steps and fine-tuning GC-OTE takes 23 hours with 60,000 steps. Though, it takes more computation for graph context based model training, the inference could be efficient if both head and tail context representations are precomputed and saved for each entity in the knowledge graph. In this section, we first present the results of link prediction, followed by the ablation study and error analysis of our models. Table 2 compares the proposed models ( OTE and graph context based GC-OTE ) to several state-of-the-art models: including translational distance based TransE (Bordes et al., 2013), RotatE (Sun et al., 2019); semantic matching based DistMult (Yang et al., 2014), ComplEx (Trouil-lon et al., 2016), ConvE (Dettmers et al., 2018), TuckER (Balazevic et al., 2019) and QuatE (Zhang et al., 2019), and graph context information based R-GCN+ (Schlichtkrull et al., 2017), SACN (Shang et al., 2019) and A2N (Bansal et al., 2019). These Model FB15k-237 WN18RR MRR H1 H3 H10 MRR H1 H3 H10 TransE .294 -.465 .226 -.501 RotatE .338 .241 .375 .533 .476 .428 .492 .571 DistMult .241 .155 .263 .419 .43 .39 .44 .49 ComplEx .247 .158 .275 .428 .44 .41 .46 .51 ConvE .325 .237 .356 .501 .43 .40 .44 .52 QuatE .348 .248 .382 .550 .488 .438 .508 .582 TurkER .358 .266 .392 .544 .470 .443 .482 .526 R-GCN+ .249 .151 .264 .417 --SACN .352 .261 .385 .536 .47 .43 .48 .54 A2N .317 .232 .348 .486 .45 .42 .46 .51 OTE .351 .258 .388 .537 .485 .437 .502 .587 GC-OTE .361 .267 .396 .550 .491 .442 .511 .583 Table 2: Link prediction for FB15k-237 and WN18RR on test sets. From Table 2, we observe that: 1) on FB15k-237, OTE outperforms RotatE , and GC-OTE outperforms all other models on all metrics. Specifically MRR is improved from 0 . 338 in RotatE , to 0 . 361 , about 7 % relative performance improvement. OTE which increases sub-embedding dimension from 2 to 20 , and graph context each contributes about half the improvement; 2) on WN18RR, OTE outperforms RotatE and GC-OTE achieves the new state-of-the-art results (as far as we know from published papers). These results show the effectiveness of the proposed OTE and graph context for the task of predicting missing links in knowledge graph. Moreover, GC-OTE improves more on FB15k-237 than on WN18RR. This is because FB15k-237 has richer graph structure context compared to WN18RR: an average of 19 edges per node v.s. 2 edges per node in WN18RR. These results indicate that the proposed method GC-OTE is more effective on data set with rich context structure information. Table 3 shows the results of ablation study of the proposed models and compares the number of model parameters with RotatE on FB15k-237 validation set. We perform the ablation study with embedding dimension of 400 . The entity embedding dimension for RotatE-S and RotatE-L are 400 and 2000 , respectively. First we notice that increasing embedding size from 400 to 2000 makes RotatE model size more than quadrupled while the performance gain is very limited (Row 1 and 2 in Table 3); increasing group embedding size from 2 to 20 does not increase the model size of OTE much, but with nice performance gain (Row 3 and 4 in Table 3). The model size of OTE is less than one-third of the size of RotatE-L but with better performance. This shows the effectiveness of the OTE . Impact of sub-embedding dimension : we fix the embedding dimension as 400 , and increase the sub-embedding dimension d s from 2 to 20 , the MRR of OTE is improved from 0.327 to 0.355 (See Row 3 and Row 4). For RotatE , the entity is embedded in complex vector space, this is similar to our setting with sub-embedding dimension = 2. Our results show that increasing the sub-dimension with OTE is beneficial to link prediction. Impact of orthogonal transform : we replace the orthogonal transform operation in OTE with two different settings, 1) removing the diagonal scalar tensor as Eq. 1 (See OTE-scalar ) and 2) using normal linear transform rather than orthogonal transform (See LNE ). Both settings lead to MRR degradation. This indicates the proposed orthogonal transform is effective in modeling the relation patterns which are helpful for link prediction. Impact of graph context : we add the graph context based model to both OTE (See GC-OTE ) and RotatE-L (See GC-RotatE-L ). We observe that MRRs are improved for both RotatE-L and OTE . This shows the importance of modeling context information for the task of link prediction. Sub-embedding dimension size: in Table 3 we show that increasing sub-embedding dimension brings a nice improvement on MRR. Is the larger size always better? Figure 2 shows the impact of d s on the OTE performance with the changing of sub-embedding size. We fix the entity embedding dimension as 400 , and vary the sub-embedding size from 2 , 5 , 10 , 20 , 50 , all the way to 100 . The blue line and green bar represent MRR and H @10 value, respectively. From Figure 2 we observe that, both MRR and Hit@10 are improved and slowly saturated around d s = 20 The similar experiments are also conducted on WN18RR data set and we find the best sub-embedding dimension is 4 on WN18RR. RotatE-L GC-OTE Type Num. H T A H T A 1-to-N 2255 .710 .169 .440 .718 .204 .461 N-to-1 5460 .156 .850 .503 .209 .863 .536 N-to-N 9763 .490 .631 .561 .508 .651 .579 Table 4: H@10 from FB15-237 validation set by categories (1-to-N, N-to-1 and N-to-N). We present error analysis of the proposed model on 1-to-N, N-to-1 and N-to-N relation predictions on FB15k-237. Table 4 shows results in terms of Hit@10, where Num. is the number of triples in the validation set belonging to the corresponding category, H/T represents the experiment to predict head entity /tail entity, and A denotes average result for both H and T. Assume c ( h, r ) and c ( r, t ) are the number of ( h, r ) and ( r, t ) pairs appeared in triples from the training set respectively. A triple ( h, r, t ) from the validation set is considered as one of the categories in the following: ( h, r, t ) =   N-to-1 , if c ( h, r ) > 1 and c ( r, t )  1 1-to-N , if c ( h, r )  1 and c ( r, t ) > 1 N-to-N , if c ( h, r ) > 1 and c ( r, t ) > 1 other. From Table 4 we observe that, comparing to RotatE large model, the proposed model get better Hit@10 on all cases, especially for the difficult cases when we attempt to predicting the head entity for 1-to-N/N-to-N relation type, and tail entity in N-to-1/N-to-N relation type. The reason is because that in the proposed model, the groupings of sub-embedding relation pairs in OTE and graph context modeling both are helpful to distinguish N different tails/heads when they share the same (head, rel)/(rel, tail). In this paper we propose a new distance-based knowledge graph embedding for link prediction. It includes two-folds. First, OTE extends the modeling of RotatE from 2D complex domain to high dimensional space with orthogonal relation transforms. Second, graph context is proposed to integrate graph structure information into the distance scoring function to measure the plausibility of the triples during training and inference. The proposed approach effectively improves prediction accuracy on the difficult N-to-1, 1-to-N and N-to-N link predictions. Experimental results on standard benchmark FB15k-237 and WN18RR show that OTE improves consistently over RotatE , the state-of-the-art distance-based embedding model, especially on FB15k-237 with many high in-degree nodes. On WN18RR our model achieves the new state-of-the-art results. This work is partially supported by Beijing Academy of Artificial Intelligence (BAAI).
abstain abstain objective objective abstain abstain abstain abstain abstain result other abstain abstain abstain abstain other abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain objective objective abstain abstain method abstain objective abstain abstain result objective other objective other other other other other other other other other other other other other method other other other other other other other other method method abstain abstain abstain method abstain abstain method abstain abstain abstain abstain other abstain abstain method abstain abstain method method abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain method method method method method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain method abstain abstain abstain abstain method abstain method method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain method abstain method method method abstain abstain abstain other other result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain result abstain abstain abstain abstain result result abstain result abstain abstain abstain abstain result abstain abstain abstain abstain abstain other abstain abstain objective abstain abstain abstain abstain abstain objective abstain