Predicting missing facts in a knowledge graph (KG) is crucial as modern KGs are far from complete. Due to labor-intensive human labeling, this phenomenon deteriorates when handling knowledge represented in various languages. In this paper, we explore multilingual KG completion, which leverages limited seed alignment as a bridge, to embrace the collective knowledge from multiple languages. However, language alignment used in prior works is still not fully exploited: (1) alignment pairs are treated equally to maximally push parallel entities to be close, which ignores KG capacity inconsistency; (2) seed alignment is scarce and new alignment identification is usually in a noisily unsupervised manner. To tackle these issues, we propose a novel self-supervised adaptive graph alignment (SS-AGA) method. Specifically, SS-AGA fuses all KGs as a whole graph by regarding alignment as a new edge type. As such, information propagation and noise influence across KGs can be adaptively controlled via relation-aware attention weights. Meanwhile, SS-AGA features a new pair generator that dynamically captures potential alignment pairs in a self-supervised paradigm. Extensive experiments on both the public multilingual DBPedia KG and newly-created industrial multilingual E-commerce KG empirically demonstrate the effectiveness of SS-AGA 1 . Knowledge graphs (KGs) like Freebase (Bollacker et al., 2008) and DBPedia (Lehmann et al., 2015) are essential for various knowledge-driven applications such as question answering (Yasunaga et al., 2021) and commonsense reasoning (Lin et al., 2021). A KG contains structured and semantic information among entities and relations, where prior  Part of work was done during internship at Amazon;  Corresponding author. knowledge can be instantiated as factual triples (head entity, relation, tail entity), e.g., ( Apple Inc. , Founded by , Steven Jobs ). As new facts are continually emerging, modern KGs are still far from being complete due to the high cost of human annotation, which spurs on the Knowledge Graph Completion (KGC) task to automatically predict missing triples to complete the knowledge graph. The KG incompletion circumstance is exacerbated in the multilingual setting, as human annotations are rare and difficult to gather, especially for low-resource languages. Unfortunately, most efforts for KGC have been devoted to learning each monolingual KG separately (Peng et al., 2021; Xu et al., 2021; Liang et al., 2021; Cao et al., 2021; Lovelace et al., 2021), which usually underperform in low-resource language KGs that suffer from the sparseness (Chen et al., 2017, 2020; Sun et al., 2020). In contrast, KGs from multiple languages are not naturally isolated, which usually share some real-world entities and relations. The transferable knowledge can be treated as a bridge to align different KGs, which not only facilitates the knowledge propagation to low-resource KGs but also alleviates 474 costly manual labeling for all languages. In this paper, we explore multilingual KG completion (MKGC) (Chen et al., 2020) with limited seed alignment across languages. To mitigate language gaps, some efforts have been initiated on multilingual KG embedding methods, which leverage a KG embedding module (e.g., TransE (Bordes et al., 2013)) to encode each language-specific KG independently and then employ an alignment loss to force pairs of aligned entities to be close maximally (Chen et al., 2020; Zhang et al., 2019; Sun et al., 2020). However, such approaches mainly involve two limitations: (1) the KG inconsistency issue among different languages is neglected due to the equal treatment for parallel entities; (2) the scarcity of seed alignment hinders the efficient knowledge transfer across languages. Concretely, prior methods treat all alignment pairs equally by forcing all parallel entities to be maximally close to each other (Chen et al., 2018; Sun et al., 2018; Chen et al., 2017). This ignores potentially negative effects from the KG inconsistency due to the language diversity. For example, as shown in Figure 1, the support English KG in DBP-5L (Chen et al., 2020) has much more enriched knowledge (80K facts) than the Greek one (13K facts). In order to complete the query ( Apple Inc. , Founded by , ?) in the resource-poor Japanese KG (28K facts), we can transfer more knowledge from resource-rich English KG through the alignment link of Steven Jobs than that of the low-data Greek. However, if roughly pushing Steven Jobs to be equally close to that English KG and Greek KG, the learned embeddings for Steven Jobs will be similar even though they have different structures, KG capacity, coverage and quality. As such, it will bring in irrelevant information regarding this query and may cause the model to get the wrong answer. Thus, we encourage the model to automatically distinguish the underlying inconsistency and transfer knowledge from suitable support KGs 2 for better language-specific KGC performance. One the other hand, seed alignment is critical for cross-lingual transfer (Chen et al., 2020; Sun et al., 2020), while acquisition of such parallel entities across languages is costly and often noisy. To mitigate such issue, some recent works (Chen et al., 2018, 2020) propose to generate new alignment pairs based on the entity embedding similarity dur-2 We regard the remaining KGs as the support KGs when conducting the KGC task in the target one. ing the training process. The generated new pairs can increase the inter-connectivity between KGs to facilitate knowledge transfer. However, simple usage of correlations between entities without any supervision may increase the noise during training, and inhibit the effectiveness of realistic language alignment in KGs (Sun et al., 2020). Motivated by these observations, we propose a S elfS upervised A daptive G raph A lignment ( SS-AGA ) framework for MKGC. To tackle the knowledge inconsistency issue, SS-AGA regards alignment as a new edge type between parallel entities instead of a loss constrain, which fuses KGs from different languages as a whole graph. Based on such unified modeling, we propose a novel GNN encoder with a relation-aware attention mechanism, which aggregates local neighborhood information with learnable attention weights and differs the influence received from multiple alignment pairs for the same entity as shown in Figure 1(b). To alleviate the scarcity of seed alignment, SS-AGA exploits a new pair generator that iteratively identifies new alignment pairs in a self-supervised manner. This is achieved by masking some seed alignment in the fused KG before GNN encoding and teaching the generation module to recover them. Empirically, SS-AGA outperforms popular baselines in both public and industrial datasets. For the public dataset, we use the multilingual DBPedia KG (Chen et al., 2020) and for the industrial dataset, we create a multilingual E-commerce Product KG called E-PKG. Our contributions are as follows: (1) We handle the knowledge inconsistency issue for MKGC by treating entity alignment as a new edge type and introducing a relation-aware attention mechanism to control the knowledge propagation; (2) We propose a new alignment pair generation mechanism with self-supervision to alleviate the scarcity of seed alignment; (3) We constructed a new industrial-level multilingual E-commerce KG dataset; (4) Extensive experiments verify the effectiveness of SS-AGA in both public and industrial datasets. A knowledge graph G = ( E , R , T ) consists of a set of entities E , relations R , and relational facts T = { ( e h , r, e t ) } , where e h , e t E are head and tail entities, and r  R is a relation. Entities and relations are represented by their text descriptions. The 475 KGC Decoder () GNN '()*+  () e /0 1 e /0 2 e 3 0 1 e 4 0 1  6 7 1 e / 0 1  3  4  6  4 e 3 0 2 e 4 0 2  6  / e 9 0 2 KG 2 KG 1 e 6 0 2 e / 0 2  9  /  3  ;<=> e 3 0 1 e 4 0 1 e 6 0 1 e / 0 1  3  4  6  4 e 3 0 2 e 4 0 2  6  / e 9 0 2 KG 2 KG 1 e 6 0 2 e / 0 2  9  /  3 KG completion task seeks to impute the missing head or tail entity of a triple given the relation and the other entity. Without loss of generality, we hereafter discuss the case of predicting missing tails, which we also refer to as a query q = ( e h , r, ? e t ) . Multilingual KG completion (MKGC) utilizes KGs across multiple languages to achieve more accurate KG completion task on each individual KG (Chen et al., 2020). Formally, we are given M different language-specific KGs as G 1 , G 2 ,    , GM , and only limited entity alignment pairs  G i  G j  { ( e i , e j ) : e i  E i , e j  E j } between G i and G j . We also call  G i  G j the seed alignment pairs to distinguish it from the new or pseudo alignment. Each KGG i has their own relation set R i . We denote the union of relation sets from all KGs as a unified relation set R = R 1 R 2   R M . MKGC is related to but different from the entity alignment (EA) task (Cao et al., 2019; Sun et al., 2020). In MKGC, seed alignment is not direct supervision while the auxiliary input features, all used in the training stage for cross-lingual transfer to boost the KGC results. KG embedding models aim to learn latent low-dimensional representations for entities { e } e E and relations { r } r R . A naive implementation is an embedding lookup table (Bordes et al., 2013; Sun et al., 2019). Recently, Graph Neural Networks (GNN) have been explored to aggregate neighborhood information in KGs, where each triple is no longer considered independent of each other (Hao et al., 2019). Mathematically, these methods employ a GNN-based encoder g that embeds entities considering the neighborhood information, { e } e E = g ( G ) . We introduce SS-AGA for MKGC, consisting of two alternating training components (a) and (b) in Figure 2: (a) A new alignment pair generation module for alleviating the limited seed alignment in G fuse . Specifically, we mask some seed alignment in the fuse KG to obtain G Maskedfuse and train the generator g a (  ) to recover them. Then, the trained generator will propose new edges based on the learned entity embeddings, which will be incorporated to G fuse as (cid:101) G fuse for MKG embedding model g k (  ) in the next iteration; (b) A novel relation-aware MKG embedding model g k (  ) for addressing the knowledge inconsistency across multilingual KGs. Specifically, we fuse different KGs as a 476 whole graph G fuse by treating alignment as a new edge type. Then g k (  ) computes the contextualized embeddings for each node with learnable relation-aware attention weights that differ the influence received from multiple alignment pairs. Finally, a KGC decoder f (  ) computes the triple scores. As mentioned before, the knowledge transfer is inefficient in existing MKGC methods, as they encode each KG separately and transfer knowledge by forcing aligned entities to share the same embedding. To handle the knowledge inconsistency, we first fuse all KGs as a whole, which relaxes the entity alignment to relational facts. We then design an attention-based relation-aware GNN to learn the contextualized MKG embeddings for entities, which can differ the influence from multiple alignment sources with learnable attention weights. Afterwards, we apply a KGC decoder on the contextualized embedding to get the triple scores for relational facts. More specifically, we create the fused KG by preserving triples within each KG and converting each cross-KG alignment pair ( e i , e j ) to two relational facts ( e i , r align , e j ) and ( e j , r align , e i ) with the alignment edge as a newly introduced relation r align . In this way, we enable direct message passing among entities from different KGs, where the attention weight can be learned automatically from data to differ the influence from multiple alignment pairs. We denote the fused knowledge graph as G fuse = ( E fuse , R fuse , T fuse ) , where E fuse = (cid:83) Mi =1 E i , R fuse ! = ( (cid:83) Mi =1 R i )  { r align } and T fuse = ( (cid:83) Mi =1 T i )  ( (cid:83) i,j { ( e h , r align , e t ) : ( e h , e t ) or ( e t , e h )   G i  G j } ) . Given the fused KGG fuse , we propose an attention-based relation-aware GNN encoder g k (  ) to learn contextualized embeddings for entities following a multi-layer message passing architecture. At the l -th layer of GNN, we first compute the relation-aware message delivered by the entity e i in a relational fact ( e i , r, e j ) as follows: h li ( r ) = Msg (cid:16) h li , r (cid:17) := W lv Concat( h li , r ) , where h li is the latent representation of e i at the l -th layer, Concat(  ,  ) is the vector concatenation function, and W lv is a transformation matrix. Then, we propose a relation-aware scaled dot product attention mechanism to characterize the importance of each entity's neighbor e i to itself e j , which is computed as follows: Att (cid:16) h li ( r ) , h lj (cid:17) = exp(  rij ) (cid:80) ( e i  ,r ) N ( e j ) exp (cid:16)  ri  j (cid:17)  rij = (cid:16) W lk h li ( r ) (cid:17) T  (cid:16) W lq h lj (cid:17)  1  d   r , (1) where d is the dimension of the entity embeddings, W lk , W lq are two transformation matrices, and  r is a learnable relation factor. Different from the traditional attention mechanism (Velickovic et al., 2018; Bai et al., 2019), we introduce  r to characterize the general significance of each relation r . It is essential as not all the relationships contribute equally to the query entity. We also remark that the neighborhood is bidirectional, i.e. N ( e j ) := { ( e i  , r ) : ( e i  , r, e j )  T fuse or ( e j , r, e i  )  T fuse } as the tail entity will also influence the head entity. We then update the hidden representation of entities by aggregating the message from their neighborhoods based on the attention score: h l +1 j = h lj +    (cid:88) ( e i  ,r ) N ( e j ) Att (cid:16) h li  ( r ) , h lj (cid:17)  h li  ( r )   , where  (  ) is a non-linear activation function, and the residual connection is used to improve the stability of GNN (He et al., 2015). Finally, we stack L layers to aggregate information from multi-hop neighbors and obtain the contextualized embedding for each entity e j as: e j = h Lj . Given the contextualized entity embeddings, the KGC decoder computes the triple score for each relational fact: f ( e h , r , e t ) . The learning object is to minimize the following hinge loss: JK = (cid:88) ( eh,r,et ) T m ( eh  ,r,et  ) / T m m =1 ,...,M (cid:2) f (cid:0) e h  , r , e  t (cid:1)  f ( e h , r , e t ) +  (cid:3) + , (2) where  > 0 is a positive margin, f is the KGC decoder, ( e h  , r, e t  ) is a negative sampled triple obtained by replacing either head or tail entity of the true triple ( e h , r, e t ) randomly by other entities in the same language-specific KG. Remark 1. Our method views cross-KG alignment as a relation r align in the fused KG. The knowledge transfer cross KGs is essentially conducted via the learnable attention weight  r align ij , where e i and e j are connected through the relation r align . Thanks to the power of GNN,  r align ij differs the influence from multiple alignment sources, as 477 opposed to some existing models that simply force pairs of entities to be close to each other through a pre-defined alignment loss. In this way, we properly conduct knowledge transfer among KGs with aware of their knowledge inconsistency. Scalability issue. Since we fuse all the M KGs as a whole, and duplicate edges for head entities, the scale of the graph G fuse would become very large. We therefore employ a k -hop graph sampler that samples the k -hop neighbors for each node and compute their contextualized embeddings. In multilingual KGs, we are only provided with limited seed alignment pairs to facilitate knowledge transfer, as they are expensive to obtain and even sometimes noisy (Sun et al., 2020). To tackle such challenge, we propose a self-supervised new alignment pair generator. In each iteration, the generator identifies new alignment pairs which will be fed into the GNN encoder g k (  ) to produce the contextualized entity embeddings in the next iteration. The training of the generator is conducted in a self-supervised manner, where the generator is required to recover masked alignment pairs. New Pair Generation (NPG) relies on two sets of entity embeddings: the structural embeddings and the textual embeddings. The structural embeddings are obtained by another GNN encoder g a : { e a } e E fuse = g a ( G fuse ) , which shares the same architecture with g k (  ) in the relation-aware MKG Embedding model (Section 3.1). The reason we employ two GNN encoders is that the set of embeddings that generate the best alignment results may differ from those that can best achieve the KG completion task. The textual embeddings are obtained by entities' text description and mBERT: e text = mBERT( e ) . mBERT is a multilingual pre-trained language model (Devlin et al., 2019) and is particularly attractive to the new alignment pair generation due to the following merits: (1) it captures rich semantic information of the text; (2) the pre-trained BERT embeddings are also aligned across different languages (Devlin et al., 2019; Sun et al., 2020). We then model the pairwise similarity score between entity e i and e j as the maximum of the co-sine similarities of their structural embeddings and textual embeddings: sim( e i , e j ) = max (cid:0) cos (cid:0) e ai , e aj (cid:1) , cos (cid:0) e text i , e text j (cid:1)(cid:1) . Then we introduce new alignment pairs if a pair of unaligned entities in two KGs are mutual nearest neighbors according to the cross-domain similarity local scaling (CSLS) measure (Conneau et al., 2018) as shown below, CSLS( e i , e j ) = 2sim( e i , e j )  s ( e i )  s ( e j ) subject to s ( e i ) = 1 K (cid:88) e i  N ( e i ) sim ( e i , e i  ) , where K is the number of each node's k-nearest neighbors. CSLS is able to capture the sturctural similarity between pairs of entities. The generated pairs are then utilized to update the graph structure of G fuse to (cid:101) G fuse in the next iteration, to alleviate the challenge of limited seed alignment. Self-Supervised Learning (SSL) Similar to many existing works (Chen et al., 2020; Sun et al., 2020), the aforementioned NPG paradigm is unsupervised and may bring in unexpected noises. Inspired by masked language modeling (Devlin et al., 2019) which captures contextual dependencies between tokens, we propose a self-supervised learning procedure to guide and denoise the new pair generation. Specifically, we randomly mask out some alignment relational facts, T masked { ( e h , r, e t )  T fuse : r = r align } , and let the generator to recover them. Such masked alignment recovery in KGs can automatically identify the underlying correlations for alignment neighbors and encourage the NPG to generate high-quality alignment pairs that are real existences but hide due to the limited seed alignment. Given the fused KG with masked alignment G Maskedfuse = {E fuse , R fuse , T fuse / T masked } , the GNN encoder g a embeds the entities as { (cid:101) e } e E fuse = g a ( G Maskedfuse ) . The GNN g a is then trained via minimizing the following hinge loss JA , JG i  G j A = (cid:88) ( eh,et )   pij ( eh  ,et  )   nij (cid:2)  (cid:101) e ah  (cid:101) e at  2   (cid:101) e ah   (cid:101) e at   2 +  a (cid:3) + JA = (cid:88) 1  i<j  MJG i  G j A , (3) where  pij = { ( e h  E i , e t  E j ) : ( e h , r align , e t )  T masked } is the masked alignment set,  nij = { ( e h  E i , e t  E j ) : ( e h , e t ) /   G i  G j } is the unaligned entity pair set, and  a > 0 is a positive margin. ( e h  , e t  ) is randomly sampled by replacing one of the entities in the positive entity pairs. The overall loss function is the combination of the KG completion loss Eq. (2) and the self-supervised alignment loss Eq. (3) as shown below J = JK +  JA , (4) where  > 0 is a positive hyperparameter to bal-ance between the two losses. We summarize the training process in Algorithm 1 of the Appendix. We conduct experiments over two real-world datasets. (i) DBP-5L (Chen et al., 2020) contains five language-specific KGs from DBpedia (Lehmann et al., 2015), i.e., English (EN), French (FR), Spanish (ES), Japanese (JA), Greek (EL). As the original dataset only contains structural information, we additionally crawled the text information for these entities and relations based on the given URLs. (ii) E-PKG is a new industrial multilingual E-commerce product KG dataset, which describes phone-related product information from an E-commerce platform across six different languages: English (EN), German (DE), French (FR), Japanese (JA), Spanish (ES), Italian (IT). The statistics are shown in Table 1. The # Aligned Links for a specific KGG i denotes the number of alignment pairs where one of the aligned entities belong to that KG. It is possible for an entity to have multiple alignment pairs across different KG sources. For both datasets, we randomly split the facts in each KG into three parts: 60 % for training, 30 % for validation, and 10 % for testing. Please refer to Appendix A for the details of E-PKG construction. In the testing phase, given each query ( e h , r, ? e t ) we compute the plausibility scores f ( e h , r, (cid:101) e t ) for triples formed by each possible tail entity (cid:101) e t in the test candidate set and rank them. We report the mean reciprocal ranks (MRR), accuracy (Hits@1) and the proportion of correct answers ranked within the top 10 (Hits@10) for testing. We also adopt the filtered setting following previous works based on the premise that the candidate space has excluded the triples that have been seen in the training set (Wang et al., 2014a; Yang et al., 2015a). Monolingual Baselines. (i) TransE (Bordes et al., 2013) models relations as translations in the Euclidean space; (ii) RotatE (Sun et al., 2019) models relations as rotations in the complex space; (iii) DisMult (Yang et al., 2015b) uses a simple bilinear formulation; (iv) KG-BERT (Yao et al., 2020) employs pre-trained language models for knowledge graph completion based on text information of relations and entities. Multilingual Baselines. (i) KEnS (Chen et al., 2020) embeds all KGs in a unified space and exploits an ensemble technique to conduct knowledge transfer; (ii) CG-MuA (Zhu et al., 2020) is a GNN-based KG alignment model with collective aggregation. We revise its loss function to conduct MKGC. (iii) AlignKGC (Singh et al., 2021) jointly trains the KGC loss with entity and relation alignment losses. For fair comparison, we use mBERT (De-vlin et al., 2019) to obtain initial embeddings of entities and relations from their text for all methods. We do not employ any pretrained tasks such as EA to obtain these initial text embeddings as in (Singh et al., 2021). The main results are shown in Table 2 and Table 3. Firstly, by comparing multilingual and monolingual KG models, we can observe that multilingual methods can achieve better performance. This indicates that the intuition behind utilizing multiple KG sources to conduct KG completion is indeed beneficial, compared with inferring each KG independently. Notably, multilingual models tend 479 Method Metric EL JA ES FR EN Monolingual Baselines TransE H@1 13.1 21.1 13.5 17.5 7.3 H@10 43.7 48.5 45.0 48.8 29.3 MRR 24.3 25.3 24.4 27.6 16.9 RotatE H@1 14.5 26.4 21.2 23.2 12.3 H@10 36.2 60.2 53.9 55.5 30.4 MRR 26.2 39.8 33.8 35.1 20.7 DisMult H@1 8.9 9.3 7.4 6.1 8.8 H@10 11.3 27.5 22.4 23.8 30.0 MRR 9.8 15.8 13.2 14.5 18.3 KG-BERT H@1 17.3 26.9 21.9 23.5 12.9 H@10 40.1 59.8 54.1 55.9 31.9 MRR 27.3 38.7 34.0 35.4 21.0 Multilingual Baselines KenS H@1 28.1 32.1 23.6 25.5 15.1 H@10 56.9 65.3 60.1 62.9 39.8 MRR ---CG-MuA H@1 21.5 27.3 22.3 24.2 13.1 H@10 44.8 61.1 55.4 57.1 33.5 MRR 32.8 40.1 34.3 36.1 22.2 AlignKGC H@1 27.6 31.6 24.2 24.1 15.5 H@10 56.3 64.3 60.9 62.3 39.2 MRR 33.8 41.6 35.1 37.4 22.3 SS-AGA H@1 30.8 34.6 25.5 27.1 16.3 H@10 58.6 66.9 61.9 65.5 41.3 MRR 35.3 42.9 36.6 38.4 23.1 Table 2: Main results on DBP-5L. to bring larger performance gains for those low-resource KGs such as Greek in DBP-5L, which is expected as low-resource KGs are far from complete and efficient external knowledge transfer can bring in potential benefits. Among multilingual models, our proposed method SS-AGA can achieve better performance in most cases across different metrics, languages, and datasets, which verifies the effectiveness of SS-AGA. To evaluate the effectiveness of our model design, we conduct ablation study by proposing the following model variants: (i) GNN applies the GNN encoder without relation modeling to each KG independently, and directly forces all alignment pairs to be close to each other as in prior works (Chen et al., 2020; Zhu et al., 2020); (ii) R-GNN is the proposed relation-aware MKG embedding model (Section 3.1), which utilizes all seed alignment to construct G fused and differs the influence from other KGs by the relation-aware attention mechanism; (iii) R-GNN + NPG conducts additional new pair generation for R-GNN; (iv) R-GNN + NPG + SSL is our proposed full model SS-AGA, which leverages SSL to guide the NPG process. We also investigate the effect of whether to share or not share the Method Metric EN DE FR JA ES IT Monolingual Baselines TransE H@1 23.2 21.2 20.8 25.1 17.2 22.0 H@10 67.5 65.5 66.9 72.7 58.4 63.8 MRR 39.4 37.4 37.5 43.6 33.0 37.8 RotatE H@1 24.2 22.3 22.1 26.3 18.3 22.5 H@10 66.8 64.3 67.1 71.9 58.9 64.0 MRR 40.0 38.2 38.0 41.8 33.7 38.1 DisMult H@1 23.8 21.4 20.7 25.9 17.9 22.8 H@10 60.1 54.5 53.5 62.6 46.2 51.8 MRR 37.2 35.4 35.1 38.0 30.9 34.8 KG-BERT H@1 24.3 21.8 22.3 26.9 18.7 22.9 H@10 66.4 64.7 67.2 72.4 58.8 63.7 MRR 39.6 38.4 38.3 44.1 33.2 37.2 Multilingual Baselines KenS H@1 26.2 24.3 25.4 33.5 21.3 25.1 H@10 69.5 65.8 68.2 73.6 59.5 64.6 MRR ---CG-MuA H@1 24.8 22.9 23.0 30.4 19.2 23.9 H@10 67.9 64.9 67.5 72.9 58.8 63.8 MRR 40.2 38.7 39.1 45.9 33.8 37.6 AlignKGC H@1 25.6 22.1 22.8 31.2 19.4 24.2 H@10 68.3 65.1 67.2 72.3 59.1 63.4 MRR 40.5 38.5 38.8 46.2 34.2 37.3 SS-AGA H@1 26.7 24.6 25.9 33.9 21.0 24.9 H@10 69.8 66.3 68.7 74.1 60.1 63.8 MRR 41.5 39.4 40.2 48.3 36.3 38.4 Table 3: Main results on E-PKG. for the SSL and KGC loss, respectively. We report the average Hits@1, Hits@10 and MRR over DBP-5L as shown in Table 4. As we can see, applying a GNN encoder to each KG independently would cause the performance drop as all aligned entities are being equally forced to be close to each other. Removing the new pair generation process would also cause a performance degradation due to the sparsity of seed alignment, which shows that iteratively proposing new alignment is indeed helpful. If the generation process is further equipped with supervision, the performance would be enhanced, which verifies the effectiveness of the self-supervised alignment loss. Finally, sharing the parameters of two GNN encoders would harm the performance. Though MKGC and entity alignment are two close-related tasks that can potentially benefit each other, the set of embeddings that produce the best alignment result do not necessarily yield the best performance on the MKGC task. We next study the effect of seed alignment number as depicted in Figure 3. Firstly, we can observe that SS-AGA consistently outperforms other multilingual models on varying alignment ratios. Secondly, 480 0.2 0.4 0.6 0.8 1.0 Japanese KG 59 60 61 62 63 64 65 66 67 H i t s @ 10 SG-KGEKEnSCG_MuAlign 0.2 0.4 0.6 0.8 1.0 Greek KG 42.5 45.0 47.5 50.0 52.5 55.0 57.5 H i t s @ 10 SG-KGEKEnSCG_MuAlign 0.2 0.4 0.6 0.8 1.0 French KG 54 56 58 60 62 64 66 H i t s @ 10 SG-KGEKEnSCG_MuAlign 0.2 0.4 0.6 0.8 1.0 Spanish KG 54 56 58 60 62 H i t s @ 10 SG-KGEKEnSCG_MuAlign 0.2 0.4 0.6 0.8 1.0 English KG 30 32 34 36 38 40 42 H i t s @ 10 SG-KGEKEnSCG_MuAlign Figure 3: Hits @ 10 with respect to different sampling ratio of seed alignment pairs. for low-resources KGs such as Japanese and Greek KGs, we can observe a sharp performance drop when decreasing the alignment ratio compared with those popular KGs such as English KG. This indicates that the knowledge transfer among different KGs is especially beneficial for those low-resources KGs, as popular KGs already contain relatively rich knowledge. However, such transfer process is heavily dependent on the seed alignment, which yields the necessity of new alignment generation process. To interpret the knowledge transfer across different KGs, we visualize the normalized average attention weight for each KG w.r.t. the attention score computed in Eq. (1) from different KG sources. We can see that for those popular KGs, they will receive the highest attention score from themselves such as English and French KGs. Although Japanese KG is low-resource, from the main results table 2, we can see that the gap improvement brought by multilingual methods is relatively small compared to another low-resource Greek KG. This indicates that Japanese KG may contain more reliable facts to facilitate missing triple predictions. However, for Greek KG, we can observe that the attention weights from other languages take the majority, which means that the performance boost in Greek KG is largely attributed to the efficient knowledge transfer from other KG sources. Knowledge graph embeddings (Bordes et al., 2013; Sun et al., 2019; Con, 2018) achieve the state-of-the-art state-of-the-art performance for KGC, which learn the latent low-dimensional representations of entities and relations. They measure triple plausibility based on varying score functions such as translation-based TransE (Bordes et al., 2013), TransH (Wang et al., 2014b); rotation-based RotatE (Sun et al., 2019) and language-model-based KG-BERT (Yao et al., 2020). Recently, GNN-based methods (Li et al., 2019; Zhang et al., 2020; Javari et al., 2020) have been proposed to capture node neighborhood information for the KGC tasks. GNN is a class of neural networks that operate on graph-structured data by passing local messages (Kipf and Welling, 2017; Velickovic et al., 2018; Xu et al., 2019; Bai et al., 2019; Huang et al., 2020, 2021; Wang et al., 2021). Specifically, they use GNN as an encoder to generate contextualized representation of entities by passing local messages (Kipf and Welling, 2017; Velickovic et al., 2018; Xu et al., 2019; Bai et al., 2019; Huang et al., 2020, 2021). Then, existing score functions are employed to generate triple scores which outperform the aforementioned methods that treat each triple independently only with the scoring function. Multilingual KG embeddings are extensions of monolingual KG embeddings that consider knowledge transfer across KGs with the use of limited seed alignment (Sun et al., 2020; Singh et al., 2021). Earlier work proposes different ways to reconcile KG embeddings for the entity alignment (EA) task: MTransE (Chen et al., 2017) learns a transformation matrix between pairs of KGs. MuGNN (Cao et al., 2019) reconciles structural differences via rule grounding. CG-MuA utilizes collective aggregation of confident neighborhood (Zhu et al., 2020). Others incorporate attribute information such as entity text (Zhang et al., 2019; Chen et al., 2018). To tackle the sparsity of seed alignment, BootEA (Sun et al., 2018) iteratively proposes new aligned pairs via bootstrapping. Zhu et al. (2017) utilizes parameter sharing to improve alignment performance. While they focus 481 T a r g e t KG Support KG Figure 4: Average attention weight learned in DBP-5L. on the EA task rather than the MKGC task that we tackle here, such techniques can be leveraged to conduct knowledge transfer among KGs. Recently, Chen et al. (2020) propose an ensemble-based approach for the MKGC task. In this paper, we view alignment as a new edge type and employ a relation-aware GNN to get the contextualized representation of entities. As such, the influence of the aligned entities is captured by the learnable attention weight, instead of assuming each alignment pair to have the same impact. We also propose a self-supervised learning task to propose new alignment pairs during each training epoch to overcome the sparsity issue of seed alignment pairs. In this paper, we propose SS-AGA for multilingual knowledge graph completion (MKGC). It addresses the knowledge inconsistency issue by fusing all KGs and utilizing a GNN encoder to learn entity embeddings with learnable attention weights that differs the influence from multiple alignment sources. It features a new pair generation conducted in a self-supervised learning manner to tackle the limited seed alignment issue. Extensive results on two real-world datasets including a newly-created E-commerce dataset verified the effectiveness of SS-AGA. Our current approach may fail to fully exploit the benefit of entity and relation texts. In the future, we plan to study more effective ways to combine text data with graph data for better model performance. We are also interested in studying MKGC where there no alignment pairs are given, which is a very practical setting and our current model is not able to deal with. Our paper proposed SS-AGA, a novel multilingual knowledge graph completion model for predicting missing triples in KGs considering their knowledge transfer. SS-AGA neither introduces any social/ethical bias to the model nor amplifies any bias in the data. We the created multilingual E-commerce product KG dataset by masking all customers'/sellers' identity and privacy. We only collect information related to products without any personal information leakage. Our model is built upon public libraries in Pytorch. We do not foresee any direct social consequences or ethical issues.
abstain abstain objective abstain objective abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain objective abstain objective abstain abstain abstain abstain method other abstain abstain other abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method abstain abstain method abstain abstain abstain method method method method method method abstain method method method method abstain method method method abstain other method abstain abstain method abstain method method method method abstain abstain abstain abstain method abstain abstain method method abstain abstain abstain method method abstain abstain other abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result result abstain other abstain abstain abstain abstain abstain abstain abstain other abstain result abstain abstain abstain abstain result other result abstain abstain abstain abstain result result abstain result other other other other other other other other other other other other other other other method other objective other objective objective abstain abstain abstain abstain method method abstain abstain abstain method method method abstain