Zhiwen Xie 1 , Guangyou Zhou 2  , Jin Liu 1  , Jimmy Xiangji Huang 3 1 School of Computer Science, Wuhan University 2 School of Computer Science, Central China Normal University 3 School of Information Technology, York University xiezhiwen@whu.edu.cn , gyzhou@mail.ccnu.edu.cn jinliu@whu.edu.cn , jhuang@yorku.ca Abstract The goal of Knowledge graph embedding (KGE) is to learn how to represent the low-dimensional vectors for entities and relations based on the observed triples. The conventional shallow models are limited to their expressiveness. ConvE (Dettmers et al., 2018) takes advantage of CNN and improves the expressive power with parameter efficient operators by increasing the interactions between head and relation embeddings. However, there is no structural information in the embedding space of ConvE, and the performance is still limited by the number of interactions. The recent KBGAT (Nathani et al., 2019) provides another way to learn embeddings by adaptively utilizing structural information. In this paper, we take the benefits of ConvE and KBGAT together and propose a Re lation-aware Inception network with joint local-global structural information for knowledge graph E mbedding (ReInceptionE). Specifically, we first explore the Inception network to learn query embedding, which aims to further increase the interactions between head and relation embeddings. Then, we propose to use a relation-aware attention mechanism to enrich the query embedding with the local neighborhood and global entity information. Experimental results on both WN18RR and FB15k-237 datasets demonstrate that ReInceptionE achieves competitive performance compared with state-of-the-art methods. Knowledge graphs (KGs) are at the core of most state-of-the-art natural language processing solutions and have been spotlighted in many real-world applications, including question answering (Hao et al., 2017), dialogue generation (He et al., 2017; Madotto et al., 2018) and machine reading comprehension (Yang and Mitchell, 2017). Typically, KGs  Corresponding author. are directed graphs whose nodes denote the entities and edges represent the different relations between entities. The structured knowledge in KGs is organized in the form of triples ( h, r, t ) , where h and t stand for the head and tail entities respectively, and r represents the relation from h to t . Although large-scale KGs (e.g., Freebase (Bollacker et al., 2008), DBpedia (Lehmann et al., 2015)) have already contained millions or even billions of triples, they are still far from complete since the emerging new knowledge appears. Knowledge graph embedding (KGE) is an effective solution to solve the incompletion problem. KGE aims to learn the low-dimensional vectors (embeddings) for entities and relations based on the observed triples in KGs. Conventional models including TransE (Bordes et al., 2013) and its numerous extensions (e.g., TransD (Ji et al., 2015), TransR (Lin et al., 2015), DistMul (Yang et al., 2015), ComplEx (Trouillon et al., 2016), etc.) have been proposed. These shallow models are limited to their expressiveness (Dettmers et al., 2018). Recently, CNN-based methods have been proposed to capture the expressive features with parameter efficient operators. ConvE (Dettmers et al., 2018) takes advantage of CNN and uses convolution filters on 2D reshapings of the head entity and relation embeddings. Through this, ConvE can increase the interactions between head and relation embeddings. Empirical results have proved that increasing the number of interactions is beneficial to the KGE task, but ConvE is still limited by the number of interactions (Jiang et al., 2019; Vashishth et al., 2020). Furthermore, ConvE does not consider the structural information. In contrast, graph-based methods are effective to aggregate neighborhood information to enrich the entity/relation representation (Schlichtkrull et al., 2018; Bansal et al., 2019; Nathani et al., 2019). Among them, KB-Figure 1: An example of relation-aware local and global information (left) and the general framework of our proposed ReInceptionE (right). GAT (Nathani et al., 2019) achieves state-of-the-art performance on various benchmark datasets via using graph attention networks (GAT) (Velickovic et al., 2018). KBGAT learns embeddings for every entity by taking all possible relations into account, which requires multiple hops of reasoning. In contrast, it can be beneficial to learn embeddings from a query-relevant subgraph of the local neighborhood and global entities. As an example shown in Figure 1, given a query ( Jack London , nationality , ? ) for Jack London , we can gather the relation-aware local neighbor ( place lived , Okaland ). The local neighbor allows us to project Jack London into the Okaland region of the embedding space, which can lead to a high score for predicting the target America , as Okaland and America are close in embedding space. Besides, we also note that a specific relation can be acted as the bridge to link the related entities. Considering the relation nationality , the related head entities { Kaneto Shiozawa , Shammi Kapoor , Will Smith ,    } and tail entities { America , Canada , Japan ,    } tend to be a set of person names and countries . These related entities act as a strong signal to judge whether a triple is valid or not. Based on the above observations, we take the benefits of ConvE and KBGAT together and propose a Re lation-aware Inception network with joint local-global structural information for knowledge graph E mbedding, and we name it ReInceptionE . In ReInceptionE, we first adapt Inception network (Szegedy et al., 2015, 2016)  a high performing convolutional neural network with carefully designed filters, to increase the interactions using multiple convolution filters with different scales, while at the same time to keep parameter efficient. Then, we construct a local neighborhood graph and a global entity graph by sharing the head and relation respectively for a given query. With the constructed graphs, we apply a relation-aware attention mechanism to aggregate the local neighborhood features and gather the global entity information to enrich the head/relation representation. Finally, we aggregate the joint local-global structural information using a fully connected layer to predict the missing links. In summary, we make the following three contributions: (1) It is the first to explore Inception network to learn query embedding which aims to further increase the interactions between head and relation embeddings; (2) We propose to use a relation-aware attention mechanism to enrich the query embedding with the local neighborhood and global entity information; (3) We conduct a series of experiments to evaluate the performance of the proposed method. Experimental results demonstrate that our method obtains competitive performance in comparison to these state-of-the-art models on both WN18RR and FB15k-237. The rest of this paper is structured as follows. In this section, we first describe the background and definition in Subsection 2.1, and Inception-based query encoder in Subsection 2.2. Then, we introduce the relation-aware local attention and global attention in Subsection 2.3 and 2.4, respectively. Finally, we describe the joint using of them in Subsection 2.5. Definition 3.1 Knowledge Graph G : A knowledge graph G = { ( h, r, t ) | ( h, r, t )  E RE} denotes a collection of triples, where E and R indicate entities and relations, respectively, h, t  E represent the head entity and tail entity, and r  R denotes the specific relation linking from the head entity h to tail entity t . Definition 3.2 Knowledge Graph Embedding : Knowledge graph embedding aims to learn embeddings of entities and relations with the valid triples in G , and then predict the missing head entity h given query (? , r, t ) or tail entity t given query ( h, r, ?) with the learned entity and relation embeddings. The framework of the proposed ReInceptionE is shown in Figure 1 (right). ReIncetionE consists of four modules: (1) Inception-based query encoder (InceptionE), which is used to transform the input query q = ( h, r, ?) into a k -dimensional vector v q ; (2) relation-aware local attention and (3) relation-aware global attention are used to capture the local neighborhood information and the global entity information; and (4) joint relation-aware attention is used to aggregate the different structural information using a fully connected layer. Finally, we compute the score for the given triple ( h, r, t ) based on the query embedding and the tail entity embedding. ConvE (Dettmers et al., 2018) is the first model to apply CNN for KGE, which uses 2D convolution operation to model the head and relation in a query. However, ConvE is limited by the number of interactions between the head and relation embeddings (Jiang et al., 2019; Vashishth et al., 2020). In this paper, we propose to employ the Inception network Figure 2: The structures of ConvE (left) and the proposed Inception-based query encoder (right). (Szegedy et al., 2015, 2016), a high performing convolutional neural network with carefully designed filters, to increase the interactions by taking the head and relation as two channels of the input. Figure 2 shows the differences between InceptionE (right) and ConvE (left). Obviously, ConvE cannot capture full interactions between the head and relation embeddings since the convolution operations in ConvE only slides on the entity or relation 2D matrices independently. On the contrary, InceptionE can increase the interactions between the head and relation embeddings using multiple convolution filters with different scales, while at the same time keep parameter efficient. As shown in Figure 2, given a query q = ( h, r, ?) , we first reshape the head and relation embeddings as 2D matrices denoted as v h and v r . Then, the 2D embeddings are viewed as two channels of the input for the Inception network. Thus, the entries at the same dimension of v h and v r are aligned over the channel dimension, which enables the convolution operations to increase the interactions between the head and relation embeddings. Specifically, We first use 1  1 convolutions to capture the direct interactions at the same dimension, which can be formulated as: v 1  1 = Relu ([ v h || v r ]   1  1 ) (1) where Relu (Glorot et al., 2011) is a non-linear activation function, || denotes the concatenation operation,  denotes the convolutional operation and  1  1 is the parameter of convolution filters with 1  1 size, v 1  1 denotes the interaction features of the first 1  1 convolutional layer. Then, filters with different sizes, such as 2  2 and 3  3 , are applied to capture high-level interaction features in various scales. Thus, we can get interaction features of the 2  2 and 3  3 convolutional layers, denoted by v 2  2 and v 3  3 , respectively. As suggested in (Szegedy et al., 2016), we use two 3  3 convolutions instead of a 5  5 convolution to capture interaction features in larger spatial filters, which is able to reduce the number of parameters. The two 3  3 convolutions are denoted as: v 2(3  3) = Relu ( Relu ( v 2(3  3) 1  1   13  3 )   23  3 ) (2) where v 2(3  3) 1  1 is the input interaction features,  1 3  3 and  2 3  3 are parameters of the two 3  3 convolution layers. Finally, the output interaction features with different scales and levels are concatenated and a fully connected layer is applied to obtain the embedding of the given query. Formally, we define the Inception-based query encoder model as: v q = Inception ( v h , v r ) = Relu ( vec ([ v 1  1 || v 2  2 || v 3  3 || v 2(3  3) ]) W ) (3) where W is the parameter of the fully connected layer. KBGAT learns embedding for every entity by taking all possible relations into account, and the embedding learning is impaired by the irrelevant neighbors. In contrast, it can be beneficial to learn embedding from a query-relevant neighborhood graph. In this subsection, we first construct a relation-aware neighborhood graph and then apply an attention mechanism to aggregate local graph structure information. For the query q = ( h, r, ?) , we denote its neighbors as N q = { n i = ( e i , r i ) | ( e i , r i , h )  G} . Note that, for each triple ( h, r, t ) , we create an inverse triple ( t, r  1 , h ) , which has also been used in (Lacroix et al., 2018; Dettmers et al., 2018). Thus, query (? , r, t ) can be converted to ( t, r  1 , ?) . And the neighbors { ( r j , e j ) | ( h, r j , e j )  G} for head entity h can be converted to a format of { ( e j , r  1 j ) | ( h, r j , e j )  G} . Thus, N q contains both the outgoing and incoming neighbors for a query q = ( h, r, ?) . Each neighbor n i = ( e i , r i )  N q is also a query with a head entity e i and a relation r i . Thus, each entity and relation in neighbor n i = ( e i , r i ) can be encoded using the Inception-based query encoder: v n i = Inception ( v e i , v r i ) (4) where v e i and v r i are the 2D embedding vectors of entity e i and relation r i . In practice, different neighbors may have different impacts for a given query. It is useful to determine the importance of each neighbor for a specific query. As an example in Figure 1, for the query ( Jack London , nationality , ? ), it is reasonable to focus on the the neighbors related to the relation nationality , such as ( Jack London , place lived , Oakland ). To this end, we use relation-aware attention mechanism to assign different importance for each neighbor and compute the relevant score for each neighbor using a non-linear activation layer: s i = LeakyRelu ( W 1 [ W 2 v q || W 3 v n i ]) (5) where W 1 , W 2 and W 3 are parameters to be trained and LeakyRelu (Maas et al., 2013) is the activation function. We then normalize the relevant scores for different neighbors using a softmax function to make it comparable across the neighbors, which is denoted as:  i = exp( s i ) (cid:80) n j N q exp( s j ) (6) Finally, we aggregate the neighborhood information according to their attention scores and apply a non-linear function to obtain the neighborhood vector. To keep more information of the original query embedding, we also apply a residual operation: v n = Relu   (cid:88) n i N q  i W 3 v n i   + W 2 v q (7) For simplification, we denote the above relation-aware attention operations as: v n = ReAtt ( V n , v q ) (8) where V n = { v n i | n i  N q } is a set of local neighobrhood vectors. The number of relation-aware local neighbors for each node (entity) varies from one to another, making the neighbor graph very sparse. The sparse nature would affect the accuracy of the embedding. In fact, a specific relation can be acted as the bridge to link the related entities. In this subsection, we construct a relation-aware head graph and tail graph by gathering all entities for relation r in the given query q = ( h, r, ?) . Intuitively, all head entities for relation r share some common type information. And the tail entities for relation r contain some implicit information about the type of the target entity t . For example in Figure 1, given the relation nationality , all heads { Kaneto Shiozawa , Shammi Kapoor , Will Smith ,    , } and tails { America , Canada , Japan ,    , } are the names of a person and a country, sharing the similar entity types. These relation-aware global heads and tails can provide some useful information for the KGE task. Thus, we construct relation-aware global head and tail graphs according to the head and tail entities of the relation. Let H r = { e i | ( e i , r, e j )  G} and T r = { e j | ( e i , r, e j )  G} denote a set of head and tail entities for relation r , respectively. For each head entity h ri  H r , we first represent it as an embedding vector v h ri . Then, we use relation-aware attention mechanism to capture the relevant information from all the relation-aware head entities, which is denoted as: v rh = ReAtt ( V rh , v q ) (9) where V rh = { v h ri | h ri  H r } is a set of entity vectors for relation-aware global entities. Similarly, we use relation-aware attention mechanism to capture global tail informations, which is computed as: v rt = ReAtt ( V rt , v q ) (10) where V rt = { v t ri | t ri  T r } is a set of entity embeddings for relation-aware global tails. Once obtained the relation-aware local neighborhood information v n and global head and tail vectors v ht and v rt , we concatenate these vectors and merge them by using a linear feed-forward layer: where W 4 and b are the parameters of the feed-forward layer. Finally, we compute the score for each triple ( h, r, t ) by applying a dot product of the query embedding v (cid:48) q and the tail embedding v t : f ( h, r, t ) = v (cid:48) Tq v t (12) To optimize the parameters in our model, we compute the probability of the tail t using a softmax function: p ( t | h, r ) = exp( f ( h, r, t )) (cid:80) ( h,r,t (cid:48) ) G (cid:48) { ( h,r,t ) } exp( f ( h, r, t (cid:48) )) (13) where  is a smoothing parameter, and G (cid:48) is a set of invalid triples created by randomly replacing the tail t with an invalid entity t (cid:48) . where ( h i , r i , t i )  G is a valid triple, and |E| is the number of valid triples in G . Datasets : We conduct experiments for KGE on two widely used public benchmark datasets : WN18RR (Dettmers et al., 2018) and FB15k-237 (Toutanova et al., 2015). WN18RR is a subset of WN18 (Bordes et al., 2013) while FB15k-237 is a subset of FB15k (Bordes et al., 2013). Since WN18 and FB15k contain a large number of inverse relations, making the triples in the test set can be obtained simply by inverting triples in the training set. To address the above problem, both WN18RR (Dettmers et al., 2018) and FB15k-237 (Toutanova et al., 2015) are generated by removing the inverse relations from WN18 and FB15k. In recent two Models WN18RR FB15k-237 MR MRR Hits@10 MR MRR Hits@10 TransE (Bordes et al., 2013)* 2300 0.243 0.532 323 0.279 0.441 DistMult (Yang et al., 2015)* 5110 0.430 0.490 512 0.281 0.446 ComplEx (Trouillon et al., 2016)* 5261 0.440 0.510 546 0.278 0.450 R-GCN+ (Schlichtkrull et al., 2018) --0.249 0.417 CACL (Oh et al., 2018) 3154 0.472 0.543 235 0.349 0.487 ConvE (Dettmers et al., 2018) 4187 0.430 0.520 244 0.325 0.501 NKGE (Wang et al., 2018) 4170 0.450 0.526 237 0.330 0.510 TransMS (Yang et al., 2019) 6523 -0.460 249 -0.445 AnyBURL (Meilicke et al., 2019) -0.470 0.552 -0.310 0.486 SACN (Shang et al., 2019) -0.470 0.540 -0.350 0.540 A2N (Bansal et al., 2019) -0.450 0.510 -0.317 0.486 GRank (Ebisu and Ichise, 2019) -0.470 0.539 -0.322 0.489 ConvR (Jiang et al., 2019) -0.475 0.537 -0.350 0.528 MuRE (Balazevic et al., 2019b) -0.475 0.554 -0.336 0.521 RotatE (Sun et al., 2019) 3340 0.476 0.571 177 0.338 0.533 QuatE (Zhang et al., 2019) 3472 0.481 0.564 176 0.311 0.495 InteractE (Vashishth et al., 2020) 5202 0.463 0.528 172 0.354 0.535 ConvKB (Nguyen et al., 2018) b 3433 0.249 0.524 309 0.243 0.421 CapsE (Nguyen et al., 2019) b 718 0.415 0.559 403 0.150 0.356 KBGAT (Nathani et al., 2019) b 1921 0.412 0.554 270 0.157 0.331 ReInceptionE (ours) 1894 0.483 0.582 173 0.349 0.528 ConvKB (Nguyen et al., 2018) a 2554 0.248 0.525 257 0.396 0.517 CapsE (Nguyen et al., 2019) a 719 0.415 0.560 303 0.523 0.593 KBGAT (Nathani et al., 2019) a 1940 0.440 0.581 210 0.518 0.626 Table 2: Link prediction results on WN18RR and FB15k-237 test sets. years, WN18RR and FB15k-237 have become the most popular datasets for the KGE task. Table 1 shows the summary statistics of the datasets. Implementations : For a test triple ( h, r, t ) , the purpose of KGE task is to predict missing links, e.g. predict tail entity t given head entity h and relation r or predict head entity h given tail entity t and relation r . To evaluate our method, three metrics are used, including Mean Rank (MR), Mean Reciprocal Rank (MRR), and Hit@10 (e.g. the accuracy in top 10 predictions). Please note that lower MR, higher MRR and Hits@10 indicate better performance. We follow the Filtered setting protocol (Bordes et al., 2013) to evaluate our model, i.e., ranking all the entities excluding the set of other true entities that appeared in training, validation and test sets. We initialize the embedding of entity and relation in our ReInceptionE model using the pre-trained embeddings with 100-dimension used in (Nguyen et al., 2019). We use Adam (Kingma and Ba, 2015) to optimize the model. The parameters of our model are selected via grid search according to the MRR on the validation set. We select the dropout rate from { 0 . 1 , 0 . 2 , 0 . 4 , 0 . 5 } , the learning rate from { 0 . 001 , 0 . 0005 , 0 . 0002 , 0 . 0001 } , the L 2 norm of parameters from { 1 e  3 , 1 e  5 , 1 e  8 } , the batch size from { 32 , 64 , 128 , 256 , 512 } and the smoothing parameter  in Equation 13 from { 1 , 5 , 10 } . Finally, the learning rate is set to 0.0002 for WN18RR and 0.0001 for FB15k-237. The L 2 norm of parameters is set to 1 e  5 . The batch size is set to 256. The dropout rate is set to 0.4 for WN18RR and 0.2 for FB15k-237. The smoothing parameter in Equation 13 is set to  = 5 . The number of filters for each convolution operation in the Inception module is set to 32. We Models WN18RR FB15k-237 MR MRR Hits@10 MR MRR Hits@10 ConvE 4187 0.430 0.520 244 0.325 0.501 KBGAT 1921 0.412 0.554 270 0.157 0.331 InceptionE 2317 0.451 0.563 215 0.334 0.518 ReInceptionE w/o N 1942 0.449 0.573 185 0.348 0.525 ReInceptionE w/o E 1809 0.412 0.569 186 0.343 0.522 ReInceptionE 1894 0.483 0.582 173 0.349 0.528 Table 3: Impact of different modules contributes the KGE task. observe that MRR performance increases slowly, starting to stagnate around 200 epochs. Finally, we train the model up to 200 epoches in the following experiments. The source codes are available at https://github.com/JuneTse/ReInceptionE . We compare our results with various state-of-the-art methods. Experimental results are summarized in Table 2. For all KGE models, a key step is to create the invalid triples to construct the negative samples. Most recently, Sun et al. (2020) investigated the inappropriate evaluation problem happened in ConvKB (Nguyen et al., 2018), CapsE (Nguyen et al., 2019) and KBGAT (Nathani et al., 2019). In fact, this issue comes from the unusual score distribution, e.g., the score function for some invalid triples gets the same values as the valid triples. Sun et al. (2020) also found that KBGAT removed the invalid triples when they appeared in the test set during negative sampling, suffering from the leakage of test triples. Therefore, we take the results (marked with the superscript b ) from (Sun et al., 2020) for ConvKB, CapsE and KBGAT. Besides, we also list the results reported in the original papers (marked with the superscript a ). From Table 2, we can see that our proposed ReInceptionE obtains competitive results compared with the state-of-the-art methods. On WN18RR dataset, the ReInceptionE achieves the best results using Hits@10 and MRR, and the second-best results using MR. On FB15k-237 dataset, the ReInceptionE obtains the second-best results using MR, and comparable results using MRR and Hits@10. Our proposed ReInceptionE is closely related to ConvE (Dettmers et al., 2018) and KBGAT (Nathani et al., 2019). Compared with ConvE, ReInceptionE achieves large performance gains on both WN18RR and FB15k-237 (ConvE vs. ReIn-ceptionE). The reason is that instead of simply concatenating the head and relation embeddings, ReInceptionE takes head and relation as two channels of the input and applies the Inception network to capture the rich interactions, which is able to learn expressive features by using filters with various scales. Unlike KBGAT, the ReInceptionE takes the (entity, relation) pair as a query and utilizes the relation-aware attention mechanism to gather the most relevant local neighbors and global entity information for the given query. The results again verify the effectiveness of the relation-aware local and global information for KGE. Some other methods have been proposed to address the KGE task, such as pLogicNet (Ou and Tang, 2019), RPJE (Niu et al., 2020), CoKE (Wang et al., 2019), TuckER (Balazevic et al., 2019a), D4-GUmbel (Xu and Li, 2019) and HAKE (Zhang et al., 2020). pLogicNet (Ou and Tang, 2019) and RPJE (Niu et al., 2020) leverage logic rules to improve the performance. CoKE (Wang et al., 2019) uses Transformer (Vaswani et al., 2017) to encode contextualized representations. HAKE (Zhang et al., 2020) embeds entities in the polar coordinate system to learn semantic hierarchies. D4-Gumbel (Xu and Li, 2019) uses the dihedral group to model relation composition. TuckER (Balazevic et al., 2019a) uses Tucker decomposition to learn tensor factorization for KGE. These methods take a series of different ways to model the KGE task. For example, logic rules play an important role to determine whether a triple is valid or not, we suspect that the performance of our proposed ReInceptionE can be further improved when taking the logic rules into account. We will leave the comparison and deep analysis in the future work. We describe the experimental results in Table 3 to investigate the impact of different modules in ReInceptionE. In Table 3, InceptionE is the baseline Models Predicting head Predicting tail 1-1 1-N N-1 N-N 1-1 1-N N-1 N-N WN18RR ConvE 0.975 0.414 0.110 0.950 0.975 0.153 0.303 0.949 InceptionE 0.976 0.587 0.128 0.957 0.952 0.231 0.482 0.957 ReInceptionE 0.976 0.586 0.152 0.961 0.976 0.272 0.494 0.958 FB15k-237 ConvE 0.303 0.590 0.137 0.400 0.272 0.088 0.845 0.545 InceptionE 0.573 0.624 0.175 0.452 0.557 0.124 0.865 0.557 ReInceptionE 0.609 0.651 0.185 0.473 0.594 0.149 0.872 0.603 Table 4: Link prediction results for each relation category on the WN18RR and FB15k-237 test sets using Hits@10. model without using relation-aware local neighbors and global entities. ReInception w/o N is the model without using relation-aware local neighbor information while ReInception w/o E is the model without using relation-aware global entity information. Besides, we also take two closely related models ConvE and KBGAT for fair comparison. From Table 3, we can see that our baseline InceptionE outperforms the closely related CNN-based model ConvE. Compared with ConvE, InceptionE is more powerful because it can capture the rich interaction features by using filters with various scales. And the ReInceptionE, which incorporates relation-aware local neighborhood and global entity information, outperforms the related graph-based model KBGAT. Table 3 also shows that the ReInceptionE outperforms InceptionE, ReInception w/o N and ReInception w/o E by a large margin on both datasets, which reconfirms our observations that relation-aware local neighbors and global entities can play different contributions for KGE. In this subsection, we present the experimental results on different relation types on WN18RR and FB15k-237 using Hits@10. We choose the closely related model ConvE, as well as InceptionE as the baselines. Following (Bordes et al., 2013), we classify the relations into four groups: one-to-one (1-1), one-to-many (1-N), many-to-one (N-1) and many-to-many (N-N), based on the average number of tails per head and the average number of heads per tail. Table 4 shows the link prediction results for each relation category. From Table 4, we find that InceptionE achieves better performance than ConvE for all relation types, indicating that increasing the number of interactions between head and relation embeddings is indeed beneficial to KGE task. Furthermore, our proposed ReInceptionE signifi-cantly outperforms ConvE and InceptionE for all relation types. In particular, ReInceptionE obtains larger improvements for complex relations, such as one-to-many, many-to-one and many-to-many. This again verifies our observations that increasing the interactions and taking the local-global structural information allows the model to capture more complex relations. In order to further analyze how relation-aware neighbors contribute to KGE task, we give two examples in Table 5. For the query ( Jack London , nationality , ? ), ReInceptionE assigns the highest attention scores for neighbors ( place lived , Oakland ), since Oakland and America are close to each other in embedding space because of other relations between them. And the top predictions for the query are a set of entities with the type of country. For the second example ( Jerry Lewls , languages , ? ), ReInceptionE assigns the very high score for neighbor ( place of birth , Newark ). This can allow us to project ( place of birth , Newark ) into the Jerry Lewis region of the embedding space, which can lead to a high score for predicting the target English Language . These examples give clear evidence of how our proposed ReInceptionE benefits the KGE task. In this paper, we propose a novel relation-aware Inception network for knowledge graph embedding, called ReInceptionE. ReInceptionE takes the benefits of ConvE and KBGAT together. The proposed method first employs Inception network to learn the query embedding, with the aim of increasing the interaction between head and relation embeddings, while at the same time to keep the parameter efficient. Then, we gather the relation-aware local neighborhood and global entity information with an attention mechanism and enrich the query embedding with the joint local-global structural information. Empirical studies demonstrate that our proposed method obtains comparative performance compared with the state-of-the-art performance on two widely used benchmark datasets WN18RR and FB15k-237. This work was supported by the National Natural Science Foundation of China under Grants 61972290 and 61972173, and also supported by the National Key R&D Program of China under Grant 2018YFC1604000. This research was also supported in part by the research grant from Natural Sciences and Engineering Research Council (NSERC) of Canada and York Research Chairs (YRC) program. We thank anonymous reviewers for their thorough review comments on this paper.
abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain abstain abstain result method abstain abstain abstain objective abstain method method method other objective method method method method other other other other method abstain abstain method abstain abstain abstain abstain method abstain abstain method abstain method method abstain abstain method abstain abstain method method method abstain abstain abstain abstain abstain abstain abstain abstain method method method abstain abstain abstain method abstain abstain abstain abstain method abstain method method method method abstain method abstain method abstain abstain abstain other abstain abstain abstain method abstain method method method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method other result abstain abstain abstain abstain abstain result result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain result abstain abstain abstain result abstain abstain abstain result abstain abstain result abstain abstain abstain abstain abstain abstain objective abstain abstain method objective other other other