## DERA: Dense Entity Retrieval for Entity Alignment in Knowledge Graphs

Zhichun Wang and Xuan Chen Beijing Normal University, Beijing, China zcwang@bnu.edu.cn

## Abstract

Entity Alignment (EA) aims to match equivalent entities in different Knowledge Graphs (KGs), which is essential for knowledge fusion and integration. Recently, embeddingbased EA has attracted significant attention and many approaches have been proposed. Early approaches primarily focus on learning entity embeddings from the structural features of KGs, defined by relation triples. Later methods incorporated entities' names and attributes as auxiliary information to enhance embeddings for EA. However, these approaches often used different techniques to encode structural and attribute information, limiting their interaction and mutual enhancement. In this work, we propose a dense entity retrieval framework for EA, leveraging language models to uniformly encode various features of entities and facilitate nearest entity search across KGs. Alignment candidates are first generated through entity retrieval, which are subsequently reranked to determine the final alignments. We conduct comprehensive experiments on both cross-lingual and monolingual EA datasets, demonstrating that our approach achieves state-of-the-art performance compared to existing EA methods.

The problem of EA has been studied for years and many approaches have been proposed. Early EA approaches rely on manually designed features to compute similarities of entities(Noy et al., 2017). Recently, embedding-based EA has attracted much attention, many approaches have been proposed and achieved promising performance. These approaches first embed entities in low-dimensional vector spaces, and then discover entity alignments based on distances of entity embeddings. There are mainly two paradigms of KG embedding, translation-baed methods and Graph Neural Network(GNN)-based methods. Translation-based methods learn entity embeddings using TransE or its extended models, including MTransE(Chen et al., 2017), JAPE(Sun et al., 2017), and BootEA(Sun et al., 2018), etc. GNN-based methods learn neighborhood-aware representations of entities by aggregating features of their neighbors, such approaches include GCNAlign(Wang et al., 2018), MuGNN(Cao et al., 2019), and AliNet(Sun et al., 2020a), etc.

## 1 Introduction

Knowledge Graphs (KGs) represent structured information of entities in various domains, which facilitates machines to handle domain knowledge. Most published KGs, such as YAGO(Rebele et al., 2016), DBpedia(Bizer et al., 2009), and WikiData(Vrandeˇ ci´ c and Krötzsch, 2014), are heterogeneous because they are either built from different data sources or by different organizations using varying terminologies. To integrate knowledge in separate KGs, it is essential to perform Entity Alignment (EA), which aims to discover equivalent entities in different KGs.

Early embedding-based methods focus on structure embedding of KGs, to further improve the EA results, some latter approaches explore entities' names and attributes as side information to enhance the entity embeddings. Names and attribute values are encoded by using character or word embedding techniques, for example in MultiKE(Zhang et al., 2019), AttrGNN(Liu et al., 2020) and CEA(Zeng et al., 2020), etc. Most recently, pre-trained language models (PLMs) have also been used to encode the names and attribute values, such as in BERT-INT(Tang et al., 2020), SDEA(Zhong et al., 2022).

Although continuous progress has been achieved in recently years, we find that there lacks a unified and effective way to encode all kinds of information of entities for EA. Most of the existing approaches encode structure information (relations) and attribute information (names, attributes, and descriptions, etc.) separately. Two kinds of information are encoded in different spaces, which are integrated before matching entities. Such EA paradigm faces both structure heterogeneity and attribute heterogeneity problems, which hinders their mutual enhancement.

Recently, the emergence of pre-trained language models has significantly enhanced the quality of text embeddings, proving highly effective in information retrieval, question answering and retrieval-augmented language modeling. Inspired by the recent development of embedding-based IR (dense retrieval), where relevant answers to a query are retrieved based on their embedding similarities, we formalize entity alignment in KGs as an entity retrieval problem. To find equivalent entities of two KGs, entities in one KG are used as queries to retrieval the most similar entities in the other KG. In this entity retrieval framework, different kinds of entities' information can be uniformly represented in textual forms, and we can leverage the advance of language models in embedding and searching similar entities.

More specifically, we make the following contributions in this work:

- · We formalize the EA problem as an entity retrieval task, and propose a language model based framework for this task. Within this framework, entities' information are uniformly transformed into textual descriptions, which are then encoded by language model based embedding model for nearest entity search between KGs.
- · We propose an entity verbalization model to generate homogenous textual descriptions of entities from their heterogeneous triples. We build a synthetic triple-to-text dataset by prompting GPT, which is used for effective training the verbalization model.
- · We design embedding models for entity retrieval and alignment reranking. The embedding model for entity retrieval encodes entities independently, which can efficiently find alignment candidates; the embedding model for alignment reranking encodes features of entity pairs, which captures the interactions of entities and guarantees the precision of alignments.
- · We conduct comprehensive experiments on five datasets, and compare our approach with

the existing EA approaches. The results show that our approach achieves state-of-the-art results.

The rest of this paper is organized as follows: Section 2 covers the preliminaries of our work, Section 3 details our proposed approach, Section 4 presents the experiments, Section 5 discusses related work, and Section 6 provides the conclusion.

## 2 Preliminaries

In this section, we introduce the problem of entity alignment in knowledge graphs, and formalize the task of dense entity retrieval for EA.

## 2.1 KGand Entity Alignment

Knowledge Graph (KG). KGs represent structural information about entities as triples having the form of ⟨ s, p, o ⟩ . A triple can be relational or attributional, a relational triple describes certain kind of relation between entities, and an attributional triple describes an attribute of an entity. In this work, we consider both relational and attributional triples in KGs. Formally, we represent a KG as G = ( E,R,A,L,T ) , where E R A , , and L are sets of entities, relations, attributes, and literals; T ⊆ ( E × R × E ) ∪ ( E × A × L ) is the sets of triples.

Entity Alignment (EA). Given two KGs G s and G t , and a set of pre-aligned entity pairs S = { ( u, v ) | u ∈ G , v s ∈ G , u t ≡ v } ( ≡ denotes equivalence), the task of entity alignment is to find new equivalent entity pairs between G s and G t .

## 2.2 Dense Entity Retrieval

In this work, we formalize EA as an entity retrieval task. Given a source KG G s and a target KG G t , entity retrieval aims to, for each entity s ∈ G s , return a ranked list of k most similar entities [ t 1 , t 2 , ..., t k ] in G t . The top-ranked entity t 1 is considered as be equivalent to the source entity s , i.e. s ≡ t 1 .

To achieve accurate entity retrieval, LM-based embedding models are leveraged in our approach to encode entities into dense vectors, and the similarities of entities are computed using their vectors:

$$f ( s, t ) = \sin ( \phi ( s ), \psi ( t ) ) \quad \quad ( 1 )$$

where ϕ ( ) · ∈ R d and ψ ( ) · ∈ R d are encoders mapping the source and target entities into ddimensional vector space, respectively. In this

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01154v1-figure-5a4c95a5-67f6-4ae4-a06c-869fd6a50231.jpg)

Figure 1: Framework of DERA.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01154v1-figure-1ec2b5dd-0a36-45f6-80a1-17e30c9276c2.jpg)

work, we will use the same encoder for source and target entities, and use dot-product for computing the similarity of entities.

balization can be formally defined as a mapping g ( N e , L e ) → s e , where s e is the textual sequence of e .

## 3 Method

In this section, we present the proposed EA framework DERA ( D ense E ntity R etrieval for entity A lignment), which is shown in Figure 1. Given two KGs to be aligned, DERA works in three main stages. (1) Entity Verbalization (EV) : this stage converts heterogeneous triples of entities into homogeneous natural language descriptions. Relations and attributes expressed in different languages will also be converted into one language. (2) Entity Retrieval (ER) : entities' textual descriptions are encoded in the same vector space. Entities are indexed using their embeddings, similar entities are retrieved based on embedding similarity to obtain alignment candidates. (3) Alignment Reranking (AR) : candidate alignments are further reranked by an reranking model to produce the final results.

## 3.1 Entity Verbalization

The purpose of entity verbalization is to convert relational and attribute triples of entities into textual descriptions in one language, which can be well encoded by a language model based embedding model. Given an entity e in a KG, let N e = { ( r , e i i ) } k i =1 be the set of neighbors and associated relations of entity e , L e = { ( a , v j j ) } m j =1 be the set of attributes and values of entity e ; here e i is an entity and r i is the relation connecting two entities, v j is the value of a j of e . Entity ver-

To get high-qualified verbalization results, we train a generative language model which takes triples as input context and generate textual descriptions as outputs. More specifically, we take open Large Language Models (LLMs) as base models, and build triple-to-text dataset to fine-tune base models.

Dataset Building. The triple-to-text dataset is built by instructing the GPT4 using a designed prompt template, which is shown in Figure 2. There are four parts in the prompt: (1) The first part is an instruction prefix to describe the task of generating triples of entities of specified type; we predefined 25 common entity types, including person, organization, movie, disease, etc. (2) The second part tells the model to generate a short and precise description of the generated triples; (3) The third part specifies the formates of generated triples and textual descriptions; (4) The fourth parts gives an example to the model.

Using the above prompt, we build a dataset contain triples and textual descriptions of 18,572 entities.

Model Training. Using the generated dataset, we fine-tune the LLMs with the next word prediction task, which is a universal approach to training LLMs. For an entity, given the sequence of triples x = ( e, r 1 , e 1 , ..., r k , e k , a 1 , v 1 , ..., a m , v m ) and its target textual description y = ( y , y 1 2 , . . . , y n ) , the training objective of EV model can be formu-

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01154v1-figure-1ef981c5-65bc-4f2a-9803-aa2d0ef39383.jpg)

## Prompt for Dataset Building

You need to generate some imaginary triples about {Entity Type} Each of these triples has a central entity. These triples contain the oneneighbors of central entity and the properties of those neighbors Finally; the properties of central entity itself are included. The type of entity can be arbitrary; as long as it matches the distribution of the real-world knowledge graph. To ensure that the data conforms to a realistic distribution; entities contain names 75% of the time and 25% of the time; the entity's name is represented by a hop string:

Then you need to generate a short and precise description of that central entity for dense retrieval based on you generating these triples\_ The generated description is required to contain some more ontological information of central entity in order to better generalize to the information retrieval model.

You need to generate according to the following format: Triples: {{Generated content}} Description: {{Generated content}} Do not output superfluous information outside the given format. Finally; let me give you an example of triples for reference, but do not copy exactly the reference examples provided to you.

{An Example}

Figure 2: Prompt for Building Training Dataset for Entity Verbalization Model.

lated as:

$$\mathcal { L } _ { \text{EV} } = - \frac { 1 } { n } \sum _ { t = 1 } ^ { n } \log P ( y _ { t } | x, y _ { < t } ; \theta ) \quad ( 2 ) \quad \text{enc} \quad \text{$\text{date}$} \quad.$$

where n is the length of y , y t ( t = 1 2 , , ..., n ) denotes the textual tokens of the sequence y , θ represents the model parameters.

In this work, we choose LLMs of 7B size as the base models of EV. More specifically, Mistral7B-Instruct-v0.2(Jiang et al., 2023) and Qwen1.57B-Chat(Bai et al., 2023) are used because they have excellent performances in small-size LLMs. QWen is used for EA tasks involving Chinese language, because it has great ability of handling Chinese texts. In the other EA tasks, Mistral model is used in EV stage. EV models are trained independent of specific EA tasks, once two EV models have been trained, their parameters are frozen and will not be changed in the following two stages.

## 3.2 Entity Retrieval

In this stage, entity embedding model is trained to encode entity descriptions into vector space, where entities are close to their equivalent counterparts. Using the entity embedding results, alignment candidates are produced based on embedding similarities of entities. In this work, we use a text embedding model as the basis, and fine-tune it with pre-aligned entities to further improve the embedding quality. More specifically, BGE(Chen et al., 2024) embedding model is used here because it achieves state-of-the-art performances on multilingual and cross-lingual retrieval tasks.

Model Training. As defined in Section 2.2, the similarity of two entities s and t is computed as the doc product of their embeddings:

$$f ( u, v ) = \phi ( u ) \cdot \phi ( v ). \text{ \quad \ \ } ( 3 )$$

Here ϕ ( ) · ∈ R d denotes the entity embedding model which maps the entity into d -dimensional vector space. Given a set of seed alignments S = { ( u, v ) | u ∈ G , v s ∈ G , u t ≡ v } , the entity embedding model in our approach is trained by minimizing the following contrastive loss:

$$\mathcal { L } _ { \text{ER} } = - \sum _ { ( u, v ) \in S } \log \frac { e ^ { f ( u, v ) } } { e ^ { f ( u, v ) } + \sum _ { v ^ { \prime } \in N _ { u } } e ^ { f ( u, v ^ { \prime } ) } } \ ( 4 )$$

where N u is a set of negative (inequivalent) entities for u .

Candidate Selection. After the entity embedding model is trained, all the entities in two KGs can be encoded as vectors in the same space. Then candidate alignments are obtained by using each source entity to retrieval nearest target entities based on their embeddings. More specifically, for each source entity u ∈ G s , a set of top-k nearest target entities in G t are retrieved, which are candidate alignments u , denoted as V u .

## 3.3 Alignment Reranking

In the entity retrieval stage, entities' descriptions are encoded independently from each other. To further improve the EA results, we design an alignment reranking model which capture the interactions of entities' features. Here a reranker built upon BERT is trained, which takes features of two entities as inputs, and predict the finegrained similarities of entity pairs. Entity pairs are restricted to the candidates generated by the entity retrieval stage, which helps our approach to control the computation costs in alignment reranking.

Let C = { ( u , V j u j ) } l j =1 be the alignment candidates, where u j is a source entity and V u j is the set of its candidate equivalent entities. We construct a dataset for training our alignment reranking model, let it be R = { ( u , v j j , N j ) } l j =1 , where ( u , v j j ) ∈ S is the pre-aligned entity pair and N j = V u j / v { j } is the set of candidate entities that are not equivalent to u j . The reranking model is trained by minimizing the following loss:

$$\mathcal { L } _ { \text{AR} } = & - \sum _ { ( u _ { j }, v _ { j }, N _ { j } ) \in R } \log \frac { e ^ { \delta ( u _ { j }, v _ { j } ) } } { e ^ { \delta ( u _ { j }, v _ { j } ) } + \sum _ { v _ { k } ^ { \prime } \in N _ { j } } e ^ { \delta ( u _ { j }, v _ { k } ^ { \prime } ) } } } \begin{array} { c } \text{ base } 1 \\ \text{Chat} ( B \\ \text{MED-} ) \\ \text{Instruc} \end{array}$$

Here δ ( u, v ) is the similarity score computed by the reranking model based on the inputs of two entities:

$$\delta \left ( u, v \right ) = \text{MLP} \left ( \text{BERT} _ { [ \text{CLS} ] } \left ( d _ { u }, d _ { v } \right ) \right ) \quad \left ( 6 \right ) \quad \text{in} \mathfrak { G }$$

where d u and d v represent the textual descriptions of u and v , respectively.

## 4 Experiments

## 4.1 Datasets

Datasets. To evaluate the performance of our approach, we conduct experiments on both crosslingual and monolingual datasets, including:

- · DBP15K(Sun et al., 2017) contains three cross-lingual EA datasets build from DBpedia, including Chinese-English (ZH-EN), Japanese-English (JA-EN), and FrenchEnglish (FR-EN).
- · D-W-15K(Sun et al., 2020b) is a monolingual EA dataset built from DBpedia and Wikipedia by using an iterative degree-based sampling method. Compared with DBP15K, D-W-15K contains KGs that are more like real-world ones.
- · MED-BBK-9K(Zhang et al., 2020) is a dataset built from two medical knowledge graphs, containing triples on diseases, symptoms, drugs, and diagnosis methods. It poses a more complex and realistic scenario for EA compared to traditional datasets like DBpedia.

Table 1 shows the detailed statistics of these datasets.

## 4.2 Training Details

We train the Entity Verbalization (EV), Entity Retrieval (ER), and Alignment Reranking (AR) models sequentially.

EV Model. In the training of EV model, we employ Deepspeed 1 with a context window length of 2048, the learning rate is set to 9 65 . e -

1 https://github.com/microsoft/

6 , and the batch size is 24 per GPU. For the base language models, we use Qwen1.5-7BChat(Bai et al., 2023) for DBP15KZH-EN and MED-BBK-9K datasets, and use Mistral-7BInstruct-v0.2(Jiang et al., 2023) for DBP15KJA-EN, DBP15KFR-EN, and D-W-15K datasets. Gradient accumulation is set to 1. To optimize memory usage and computation speed, we utilize Zero-Stage3(Rajbhandari et al., 2020), gradient checkpointing(Chen et al., 2016), and flash attention 2(Dao, 2023). The model is trained on 8 NVIDIA A800 GPU for 3 epochs using the AdamW optimizer.

ER Model. In the training of ER model, for each positive entity, 64 negative entities are randomly sampled from the top-200 nearest ones. The learning rate is set to 1 e -5 , and the batch size to 16. We utilize distributed negative sample sharing and gradient checkpointing(Chen et al., 2016), evaluate the model every 20 steps and saving the best model based on the MRR metric on the validation set. Training is performed on 2 NVIDIA A800 GPUs for 5 epochs.

AR Model. In the training of AR model, for each positive entity, 110 negative entities are randomly sampled from the top-200 nearest ones. The maximum text length is set to 512; the learning rate to 1 e -5 , and the batch size to 12 per GPU. Gradient accumulation steps are set to 8. We enable gradient checkpointing, evaluate the model every 10 steps, and save the best model based on the Hits@1 metric on the validation set. Training is carried out on 2 NVIDIA A800 GPUs for 5 epochs.

## 4.3 Results on DBP15K

We compare our approach with four groups of baselines on DBP15K datasets, which are categorized by the used side information: (1) approaches using attributes as side information, including JAPE(Sun et al., 2017), GCN-Align(Wang et al., 2018), JarKA(Chen et al., 2020); (2) approaches using entity names as side information, including GMNN(Xu et al., 2019), SelfKG(Liu et al., 2022) and TEA-NSP, TEA-MLM(Zhao et al., 2023); (3) approaches using attributes and names as side information, including HMAN(Yang et al., 2019), AttrGNN(Liu et al., 2020), BERT-INT(Tang et al., 2020), ICLEA(Zeng et al., 2022) and TEA-NSP, TEA-MLM(Zhao et al., 2023); (4) approaches using translated entity names as side information, including HGCN-JE(Wu et al., 2019b),

Table 1: Statistics of Experimental Datasets

| Dataset      | Language   | Entities   | Relations   | Attributes   | Rel. Triples   | Attr. Triples   |
|--------------|------------|------------|-------------|--------------|----------------|-----------------|
| DBP15K ZH-EN | ZH         | 19,388     | 1,701       | 8,113        | 70,414         | 379,684         |
|              | EN         | 19,572     | 1,323       | 7,173        | 95,142         | 567,755         |
| DBP15K JA-EN | JA         | 19,814     | 1,299       | 5,882        | 77,214         | 354,619         |
| DBP15K JA-EN | EN         | 19,780     | 1,153       | 6,066        | 93,484         | 497,230         |
| DBP15K       | FR         | 19,661     | 903         | 4,547        | 105,998        | 354,619         |
| FR-EN        | EN         | 19,993     | 1,208       | 6,422        | 115,722        | 497,230         |
| D-W-15K-V2   | EN         | 15,000     | 167         | 175          | 73,983         | 66,813          |
|              | EN         | 15,000     | 121         | 457          | 83,365         | 175,686         |
| MED-BBK-9K   | ZH         | 9,162      | 32          | 19           | 158,357        | 11,467          |
|              | ZH         | 9,162      | 20          | 21           | 50,307         | 44,987          |

RDGCN(Wu et al., 2019a), NMN(Wu et al., 2020), DATTI(Mao et al., 2022a), SEU(Mao et al., 2021), EASY(Ge et al., 2021), CPL(Ding et al., 2022), UED(Luo and Yu, 2022) and LigthEA(Mao et al., 2022b). Our approach is compared to baselines in each group using the same inputs as them. Table 2 outlines the results of all the approaches on DBP15K datasets. The best results in each group are highlighted in boldface, the second best results are highlighted with underlines.

in this group also employ optimal transport strategies to draw final alignments from entity similarities, which can effectively promote the results. To be fairly compared with these approach, we also report the results of our approach with the optimal transport strategy. According to the results, our approach gets the best results among all the approaches in this group. Among approaches without optimal transport strategies, our approach also gets the best results.

Attributes as Side Information. Approaches in this group align entities based on relations and attributes in KGs. Compared with approaches in this group, our approach obtains significantly better results, with average improvements of 25.3% of Hits@1 and 20.7% of MRR over the second best approach on three datasets.

Names as Side Information. Approaches in this group use entity names and relations to discover equivalent entities. Our approach gets the best results of Hits and MRR on ZH-EN and FR-EN datasets, it obtains 1.5% and 1.4% improvements of Hits@1 over the second best approach TEAMLM. While on the JA-EN dataset, TEA-NSP gets slightly better results than ours.

Names and Attributes as Side Information. When using both names and attributes, our approach still obtain top-ranked results. Except for the Hits@10 on JA-EN and Hits@1 on FR-EN datasets, our approach gets the best results among all the compared approaches in this group.

Translated Names as Side Information. Approaches in this group use machine translation tool to convert non-English names into English ones, and takes translated names as side information. Some of the approaches (annotated with † )

## 4.4 Results of Hard Setting on DBP15K

In the work of AttrGNN(Liu et al., 2020), a hard setting of evaluations on DBP15K was proposed. The purpose of this hard setting is to build more difficult testing set on DBP15K. Specifically, similarities of equivalent entities in the datasets are first measured using embeddings of their names, 60% entity pairs with the lowest similarities are selected as the testing set, and the remaining entity pairs are randomly split into training set (30%) and validation set (10%).

Table 3 shows that results of hard setting on DBP15K. Our approach is compared with eight baselines, including JAPE (Sun et al., 2017), BootEA(Sun et al., 2018), GCN-Align(Wang et al., 2018), MuGNN(Cao et al., 2019), MultiKE(Zhang et al., 2019), RDGCN(Wu et al., 2019a), AttrGNN(Liu et al., 2020), and FGWEA(Tang et al., 2023). According to the results, our approach DERA gets the best Hits and MRR on all of the three datasets. Figure 3 compares the Hits@10 of approaches in regular setting and hard setting on DBP15KZH\_EN. All the baselines have significant decreases of Hits@10, while DERA (using names and attributes as side

Table 2: Results on DBP15K Datasets

| Info.             | Model         | DBP15K-ZH-EN   | DBP15K-ZH-EN   | DBP15K-ZH-EN   | DBP15K-JA-EN   | DBP15K-JA-EN   | DBP15K-JA-EN   | DBP15K-FR-EN   | DBP15K-FR-EN   | DBP15K-FR-EN   |
|-------------------|---------------|----------------|----------------|----------------|----------------|----------------|----------------|----------------|----------------|----------------|
| Info.             | Model         | Hits@1         | Hits@10        | MRR            | Hits@1         | Hits@10        | MRR            | Hits@1         | Hits@10        | MRR            |
| Attributes        | JAPE          | 0.412          | 0.745          | 0.490          | 0.363          | 0.685          | 0.476          | 0.324          | 0.667          | 0.430          |
| Attributes        | GCN-Align     | 0.413          | 0.744          | 0.549          | 0.399          | 0.745          | 0.546          | 0.373          | 0.745          | 0.532          |
| Attributes        | JarKA         | 0.706          | 0.878          | 0.766          | 0.646          | 0.855          | 0.708          | 0.704          | 0.888          | 0.768          |
| Attributes        | DERA(Ours)    | 0.946          | 0.982          | 0.961          | 0.921          | 0.959          | 0.937          | 0.949          | 0.985          | 0.964          |
| Names             | GMNN          | 0.679          | 0.785          | -              | 0.740          | 0.872          | -              | 0.894          | 0.952          | -              |
| Names             | SelfKG        | 0.745          | 0.866          | -              | 0.816          | 0.913          | -              | 0.957          | 0.992          | -              |
| Names             | TEA-NSP       | 0.815          | 0.953          | 0.870          | 0.890          | 0.967          | 0.920          | 0.968          | 0.995          | 0.980          |
| Names             | TEA-MLM       | 0.831          | 0.957          | 0.880          | 0.883          | 0.966          | 0.910          | 0.968          | 0.994          | 0.980          |
| Names             | DERA(Ours)    | 0.846          | 0.962          | 0.900          | 0.866          | 0.951          | 0.889          | 0.980          | 0.996          | 0.987          |
| Names &Attributes | HMAN          | 0.871          | 0.987          | -              | 0.935          | 0.994          | -              | 0.973          | 0.998          | -              |
| Names &Attributes | AttrGNN       | 0.796          | 0.929          | 0.845          | 0.783          | 0.921          | 0.834          | 0.919          | 0.978          | 0.910          |
| Names &Attributes | BERT-INT      | 0.968          | 0.990          | 0.977          | 0.964          | 0.991          | 0.975          | 0.992          | 0.998          | 0.995          |
| Names &Attributes | ICLEA         | 0.884          | 0.972          | -              | 0.924          | 0.978          | -              | 0.991          | 0.999          | -              |
| Names &Attributes | TEA-NSP       | 0.941          | 0.983          | 0.960          | 0.941          | 0.979          | 0.960          | 0.979          | 0.997          | 0.990          |
| Names &Attributes | TEA-MLM       | 0.935          | 0.982          | 0.950          | 0.939          | 0.978          | 0.950          | 0.987          | 0.996          | 0.990          |
| Names &Attributes | DERA(Ours)    | 0.968          | 0.994          | 0.979          | 0.967          | 0.992          | 0.978          | 0.989          | 0.999          | 0.995          |
| Translated Names  | HGCN-JE       | 0.720          | 0.857          | -              | 0.766          | 0.897          | -              | 0.892          | 0.961          | -              |
| Translated Names  | RDGCN         | 0.708          | 0.846          | 0.746          | 0.767          | 0.895          | 0.812          | 0.886          | 0.957          | 0.911          |
| Translated Names  | NMN           | 0.733          | 0.869          | -              | 0.785          | 0.912          | -              | 0.902          | 0.967          | -              |
| Translated Names  | DERA(Ours)    | 0.930          | 0.982          | 0.950          | 0.917          | 0.978          | 0.941          | 0.972          | 0.995          | 0.982          |
| Translated Names  | DATTI †       | 0.890          | 0.958          | -              | 0.921          | 0.971          | -              | 0.979          | 0.990          | -              |
| Translated Names  | SEU †         | 0.900          | 0.965          | 0.924          | 0.956          | 0.991          | 0.969          | 0.988          | 0.999          | 0.992          |
| Translated Names  | EASY †        | 0.898          | 0.979          | 0.930          | 0.943          | 0.990          | 0.960          | 0.980          | 0.998          | 0.990          |
| Translated Names  | CPL-OT †      | 0.927          | 0.964          | 0.940          | 0.956          | 0.983          | 0.970          | 0.990          | 0.994          | 0.990          |
| Translated Names  | UED †         | 0.915          | -              | -              | 0.941          | -              | -              | 0.984          | -              | -              |
| Translated Names  | LightEA †     | 0.952          | 0.984          | 0.964          | 0.981          | 0.997          | 0.987          | 0.995          | 0.998          | 0.996          |
| Translated Names  | DERA † (Ours) | 0.985          | 0.997          | 0.990          | 0.994          | 0.999          | 0.996          | 0.996          | 0.999          | 0.997          |

Approaches with † employ optimal transport strategy.

information) has only 0.1% decrease, showing its remarkable robustness.

## 4.5 Results on DW-15K and MED-BBK-9K

DW15K and MED-BBK-9K are two challenging datasets of entity alignment. DW-15K is built from Wikipedia, where entity names are replaced with ids; there are also significant missing and corrupted attribute values. The dataset of MEDBBK-9K is built from an authoritative medical KG and a KG built from a Chinese online encyclopedia (Baidu Baike); many entities in MEDBBK-9K lack names and attributes, which makes the EA task more difficult. We compared our approach with seven approaches, three of them are probabilistic ones including LogMap(JiménezRuiz and Cuenca Grau, 2011), PARIS(Suchanek et al., 2011), and PRASE(Qi et al., 2021); four of them are embedding-based ones including MultiKE(Zhang et al., 2019), BootEA(Sun et al., 2018), RSNs(Guo et al., 2019) and FGWEA(Tang et al., 2023). Following the same evaluation settings of SOTA approaches on these two datasets, we report the Precision, Recall and F1 of all the compared approaches.

Table 4 outlines the results. Our approach DERA obtains 98.2% and 84.1% F1 scores on D-W-15K-V2 and MED-BBK-9K, respectively. Compared to the former best approach FGWEA, DERA gets 5.5% and 1.8% improvements of F1 scores on two datasets, respectively. It demonstrates DERA's superior performances on difficult EA tasks.

## 4.6 Ablation Study

To analyze the effectiveness and contribution of each component in the proposed approach, we conduct ablation studies on DBP15K datasets. We ran two groups of experiments, one group uses attributes as side information, and the other group uses both names and attributes as side information. In each group, we ran three variations of DERA: 1)

Figure 3: Hits@10 (%) of approaches under the regular setting and the hard setting on DBP15k.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01154v1-figure-150d7c52-b872-4d62-b35c-22953a0ee4ff.jpg)

Table 3: Results of Hard Setting on DBP15K

| Model      | DBP15K-ZH-EN   | DBP15K-ZH-EN   | DBP15K-ZH-EN   | DBP15K-JA-EN   | DBP15K-JA-EN   | DBP15K-JA-EN   | DBP15K-FR-EN   | DBP15K-FR-EN   | DBP15K-FR-EN   |
|------------|----------------|----------------|----------------|----------------|----------------|----------------|----------------|----------------|----------------|
| Model      | Hits1          | Hits10         | MRR            | Hits1          | Hits10         | MRR            | Hits1          | Hits10         | MRR            |
| JAPE       | 0.350          | 0.566          | 0.451          | 0.311          | 0.520          | 0.410          | 0.253          | 0.483          | 0.361          |
| BootEA     | 0.513          | 0.746          | 0.593          | 0.493          | 0.746          | 0.578          | 0.513          | 0.769          | 0.603          |
| GCN-Align  | 0.366          | 0.647          | 0.464          | 0.339          | 0.653          | 0.448          | 0.303          | 0.637          | 0.414          |
| MuGNN      | 0.406          | 0.746          | 0.521          | 0.399          | 0.753          | 0.515          | 0.407          | 0.783          | 0.531          |
| MultiKE    | 0.279          | 0.352          | 0.306          | 0.482          | 0.557          | 0.509          | 0.647          | 0.695          | 0.665          |
| RDGCN      | 0.604          | 0.766          | 0.662          | 0.682          | 0.838          | 0.737          | 0.829          | 0.931          | 0.866          |
| AttrGNN    | 0.662          | 0.818          | 0.719          | 0.774          | 0.903          | 0.821          | 0.886          | 0.956          | 0.912          |
| FGWEA      | 0.756          | 0.868          | 0.796          | 0.788          | 0.897          | 0.828          | 0.983          | 0.997          | 0.988          |
| DERA(Ours) | 0.967          | 0.993          | 0.977          | 0.959          | 0.992          | 0.973          | 0.987          | 1.000          | 0.993          |

DEAR without EV module, triples of entities are directly serialized to generate inputs of ER module; 2) DERA without AR module, the final alignments are returned based on the similarities computed by ER module; 3) DERA without EV and AR module. The results of the above variations of DERA are compared to the original version of DERA,changes in results are shown in small numbers after the results.

that EV module is important in handling language heterogeneity in EA tasks.

Table 5 shows the results of ablation study. According to the results, we have the following observations:

(1) Removing the EV module in DERA leads to average 1.6% decrease of Hits@1 and 1.5% decrease of MRR when using attributes as side information. The average decreases become 3.6% of Hits@1 and 3.2% of MRR when using attributes and names as side information. It shows that EV module has positive effects on the EA results. The performance decreases more on ZH-EN and JAEN datasets, where the involving languages are more different than the FR-EN dataset. It indicates

(2) Removing the ER module in DERA leads to average 1.3% decrease of Hits@1 and 1.0% of MRR when using attributes on three datasets. If attributes and names are all used as side information, DERA without AR module gets 1.5% decrease of Hits@1 and 1.1% decrease of MRR on ZH-EN and JA-EN datasets, 0.2% and 0.1% improvements of Hits@1 and MRR on FR-EN dataset. It shows that AR module works more effectively on EA tasks with high heterogeneity and linguistic differences. When the alignment results are already good enough (e.g. &gt;99% Hits@1 on FR-EN dataset), it is difficult for AR module to further improve the results.

(3) Removing both AR and RR modules in DERA leads to significant performance drops on all the datasets, there are average 4.7% decrease of Hits@1 and 3.9% decrease of MRR when attributes are used as side information. The decreases become 8.3% of Hits@1 and 6.8% of

Table 4: Results on DW-15K and MED-BBK-9K Datasets

| Model         | DW-15K-V2   | DW-15K-V2   | DW-15K-V2   | MED-BBK-9K   | MED-BBK-9K   | MED-BBK-9K   |
|---------------|-------------|-------------|-------------|--------------|--------------|--------------|
|               | Precision   | Recall      | F1          | Precision    | Recall       | F1           |
| LogMap        | -           | -           | -           | 86.4         | 44.1         | 58.4         |
| PARIS         | 95.0        | 85.0        | 89.7        | 77.9         | 36.7         | 49.9         |
| PRASE         | 94.8        | 90.0        | 92.3        | 83.7         | 61.9         | 71.1         |
| MultiKE       | 49.5        | 49.5        | 49.5        | 41.0         | 41.0         | 41.0         |
| BootEA        | 82.1        | 82.1        | 82.1        | 30.7         | 30.7         | 30.7         |
| RSNs          | 72.3        | 72.3        | 72.3        | 19.5         | 19.5         | 19.5         |
| FGWEA †       | 95.2        | 90.3        | 92.7        | 93.9         | 73.2         | 82.3         |
| DERA † (Ours) | 98.2        | 98.2        | 98.2        | 84.1         | 84.1         | 84.1         |

Table 5: Results of Ablation Study

|            | EV   | ER   | AR   | DBP15K-ZH-EN    | DBP15K-ZH-EN    | DBP15K-JA-EN    | DBP15K-JA-EN    | DBP15K-FR-EN    | DBP15K-FR-EN    |
|------------|------|------|------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|
|            | EV   | ER   | AR   | Hits@1          | MRR             | Hits@1          | MRR             | Hits@1          | MRR             |
|            | ✓    | ✓    | ✓    | 0.946           | 0.962           | 0.923           | 0.940           | 0.949           | 0.963           |
|            | ×    | ✓    | ✓    | 0.926 0 . 020 ↓ | 0.943 0 . 019 ↓ | 0.914 0 . 009 ↓ | 0.928 0 . 012 ↓ | 0.931 0 . 018 ↓ | 0.948 0 . 015 ↓ |
| Attributes | ✓    | ✓    | ×    | 0.927 0 . 019 ↓ | 0.948 0 . 014 ↓ | 0.903 0 . 020 ↓ | 0.924 0 . 016 ↓ | 0.948 0 . 001 ↓ | 0.963 0 . 000 - |
|            | ×    | ✓    | ×    | 0.892 0 . 054 ↓ | 0.918 0 . 044 ↓ | 0.859 0 . 064 ↓ | 0.885 0 . 055 ↓ | 0.927 0 . 022 ↓ | 0.946 0 . 017 ↓ |
| Names      | ✓    | ✓    | ✓    | 0.968           | 0.979           | 0.967           | 0.978           | 0.989           | 0.994           |
|            | ×    | ✓    | ✓    | 0.926 0 . 042 ↓ | 0.945 0 . 034 ↓ | 0.909 0 . 058 ↓ | 0.923 0 . 055 ↓ | 0.980 0 . 009 ↓ | 0.988 0 . 006 ↓ |
| Attrs.&    | ✓    | ✓    | ×    | 0.955 0 . 013 ↓ | 0.970 0 . 009 ↓ | 0.950 0 . 017 ↓ | 0.965 0 . 013 ↓ | 0.991 0 . 002 ↑ | 0.995 0 . 001 ↑ |
|            | ×    | ✓    | ×    | 0.883 0 . 085 ↓ | 0.911 0 . 068 ↓ | 0.812 0 . 155 ↓ | 0.848 0 . 130 ↓ | 0.980 0 . 009 ↓ | 0.988 0 . 006 ↓ |

MRR when attributes and names are used. Comparing with DERA variation with EV and ER module, DERA also have significant performance drops, which shows that EV module is necessary for obtaining good results.

## 5 Related Work

## 5.1 Embedding-based EA

of attributes in KGs. GCN-Align (Wang et al., 2018) encodes attribute information of entities into their embeddings by using GCNs. MultiKE (Zhang et al., 2019) uses a framework unifying the views of entity names, relations and attributes to learn embeddings for aligning entities. CEA (Zeng et al., 2020) combines structural, semantic and string features of entities, which are integrated with dynamically assigned weights.

Embedding-based KG alignment approaches employ TransE and GNN to learn entities' embeddings, and then find equivalent entities in the vector spaces. Early approaches mainly rely on the structure information in KGs to find alignments, including TransE-based approaches MTransE (Chen et al., 2017), IPTransE (Zhu et al., 2017), BootEA (Sun et al., 2018), etc, and GNNbased approaches MuGNN (Cao et al., 2019), NAEA (Zhu et al., 2019), RDGCN (Wu et al., 2019a) and AliNet (Sun et al., 2020a), etc. To get improved results, some approaches utilize entity attributes or names in KGs. JAPE (Sun et al., 2017) performs attribute embedding by Skip-Gram model which captures the correlations

## 5.2 Language Model-based EA

As Pre-trained Language Models(PLMs) being successfully used in various tasks, some approaches utilize PLMs to model the semantic information of entities in the task of KG alignment. AttrGNN(Liu et al., 2020) uses BERT to encode attribute features of entities. It encode each attribute and value separately, and then uses a graph attention network to compute the weighted average of attributes and values. BERTINT(Tang et al., 2020) embeds names, descriptions, attributes and values of entities using a LM; pair-wise neighbor-view and attribute-view interactions are performed to get the matching score of entities. The interactions are timeconsuming, thus BERT-INT cannot scale to large KGs. SDEA(Zhong et al., 2022) find-tunes BERT to encode attribute values of an entity into attribute embedding; attribute embeddings of neighbors are fed to BiGRU to get relation embedding of an entity. TEA(Zhao et al., 2023) sorts triples in alphabetical order by relations and attributes to form sequences, and uses a textual entailment framework for entity alignment. TEA takes entity-pair sequence as the input of PLM, and let the PLM to predict the probability of entailment. It takes pairwise input, cannot scale to large KGs. AutoAlign(Zhang et al., 2023) gets attribute character embeddings and predicate-proximity-graph embeddings by using large language models. AttrGNN, BERT-INT and SDEA use BERT to encode attribute information of entities, and then employ GNNs to incorporate relation information into entities' embeddings. Being different from these approaches, our approach directly use language model to encode both the attributes and relations of entities. TEA uses similar way to encode attribute and relation information, but it takes entity pair as input, which cannot scale to large-scale KG alignment tasks.

As the advent of Large Language Models (LLMs), there are several approaches exploring LLMs for EA. LLMEA(Yang et al., 2024) fuses the knowledge from KGs and LLMs to predict entity alignments. It first uses RAGAT to learn entity embeddings which are used to draws alignment candidates; it then uses candidate alignments as options to generates multi-choice questions, which are passed to LLMs to predict the answer. ChatEA(Jiang et al., 2024a) first uses SimpleHHEA(Jiang et al., 2024b) to obtain candidate alignments, and then leverages LLMs' reasoning abilities to predict the final results. LLMEA and ChatEA all explore the reasoning abilities of LLM to predict entity alignments. Because the number of potential alignments are usually huge, they use exiting EA methods to generate alignment candidates, from which LLMs are used to select the final results. According to the results, the improvements contributed by LLMs are restricted.

## 6 Conclusion

In this paper, we propose a dense entity retrieval approach, DERA, for entity alignment in knowledge graphs. DERA first converts entity triples into unified textual descriptions using an entity verbalization model, and then trains a language model-based embedding model to encode the entities. Candidate alignments are identified based on their similarities in the embedding space and are further reranked by an alignment reranking model. Experiments demonstrate that DERA achieves state-of-the-art results on entity alignment tasks of varying difficulty levels.

## Limitations

The primary limitation of DERA is its pipelined framework, where models in its three stages are trained sequentially. Consequently, the component models in DERA are not optimized jointly during training. Exploring efficient methods for the joint learning of these models would be a valuable direction for future work, potentially enhancing the results further. Additionally, DERA consumes more GPU power than traditional models, which is another limitation.

## References

Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng Xu, Jin Xu, An Yang, Hao Yang, Jian Yang, Shusheng Yang, Yang Yao, Bowen Yu, Hongyi Yuan, Zheng Yuan, Jianwei Zhang, Xingxuan Zhang, Yichang Zhang, Zhenru Zhang, Chang Zhou, Jingren Zhou, Xiaohuan Zhou, and Tianhang Zhu. 2023. Qwen technical report.

Christian Bizer, Jens Lehmann, Georgi Kobilarov, Sören Auer, Christian Becker, Richard Cyganiak, and Sebastian Hellmann. 2009. Dbpediaa crystallization point for the web of data. Web Semantics: science, services and agents on the world wide web, 7(3):154-165.

Yixin Cao, Zhiyuan Liu, Chengjiang Li, Zhiyuan Liu, Juanzi Li, and Tat-Seng Chua. 2019. Multi-channel graph neural network for entity alignment. In Proceedings of the 57th Annual Meeting of the Association for Computational

Linguistics, pages 1452-1461, Florence, Italy. Association for Computational Linguistics.

Bo Chen, Jing Zhang, Xiaobin Tang, Hong Chen, and Cuiping Li. 2020. Jarka: Modeling attribute interactions for cross-lingual knowledge alignment. In Advances in Knowledge Discovery and Data Mining, pages 845-856, Cham. Springer International Publishing.

Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. Bge m3embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through selfknowledge distillation.

Muhao Chen, Yingtao Tian, Mohan Yang, and Carlo Zaniolo. 2017. Multilingual knowledge graph embeddings for cross-lingual knowledge alignment. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (AAAI2017), pages 1511-1517.

Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. 2016. Training deep nets with sublinear memory cost.

Tri Dao. 2023. Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.08691.

Qijie Ding, Daokun Zhang, and Jie Yin. 2022. Conflict-aware pseudo labeling via optimal transport for entity alignment. In 2022 IEEE International Conference on Data Mining (ICDM), pages 915-920.

Congcong Ge, Xiaoze Liu, Lu Chen, Baihua Zheng, and Yunjun Gao. 2021. Make it easy: An effective end-to-end entity alignment framework. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '21, pages 777-786, New York, NY, USA. Association for Computing Machinery.

Lingbing Guo, Zequn Sun, and Wei Hu. 2019. Learning to exploit long-term relational dependencies in knowledge graphs. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 2505-2514. PMLR.

Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. Mistral 7b.

Xuhui Jiang, Yinghan Shen, Zhichao Shi, Chengjin Xu, Wei Li, Zixuan Li, Jian Guo, Huawei Shen, and Yuanzhuo Wang. 2024a. Unlocking the power of large language models for entity alignment.

Xuhui Jiang, Chengjin Xu, Yinghan Shen, Yuanzhuo Wang, Fenglong Su, Zhichao Shi, Fei Sun, Zixuan Li, Jian Guo, and Huawei Shen. 2024b. Toward practical entity alignment method design: Insights from new highly heterogeneous knowledge graph datasets. In Proceedings of the ACM on Web Conference 2024, WWW '24, pages 2325-2336, New York, NY, USA. Association for Computing Machinery.

Ernesto Jiménez-Ruiz and Bernardo Cuenca Grau. 2011. Logmap: Logic-based and scalable ontology matching. In Proceedings of the Tenth International Semantic Web Conference(ISWC2011), pages 273-288.

Xiao Liu, Haoyun Hong, Xinghao Wang, Zeyi Chen, Evgeny Kharlamov, Yuxiao Dong, and Jie Tang. 2022. Selfkg: Self-supervised entity alignment in knowledge graphs. In Proceedings of the ACM Web Conference 2022, WWW '22, pages 860-870, New York, NY, USA. Association for Computing Machinery.

Zhiyuan Liu, Yixin Cao, Liangming Pan, Juanzi Li, Zhiyuan Liu, and Tat-Seng Chua. 2020. Exploring and evaluating attributes, values, and structures for entity alignment. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6355-6364, Online. Association for Computational Linguistics.

Shengxuan Luo and Sheng Yu. 2022. An accurate unsupervised method for joint entity alignment and dangling entity detection. In Findings of the Association for Computational Linguistics:

ACL 2022, pages 2330-2339, Dublin, Ireland. Association for Computational Linguistics.

Xin Mao, Meirong Ma, Hao Yuan, Jianchao Zhu, ZongYu Wang, Rui Xie, Wei Wu, and Man Lan. 2022a. An effective and efficient entity alignment decoding algorithm via third-order tensor isomorphism. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5888-5898, Dublin, Ireland. Association for Computational Linguistics.

Xin Mao, Wenting Wang, Yuanbin Wu, and Man Lan. 2021. From alignment to assignment: Frustratingly simple unsupervised entity alignment. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2843-2853, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.

Xin Mao, Wenting Wang, Yuanbin Wu, and Man Lan. 2022b. LightEA: A scalable, robust, and interpretable entity alignment framework via three-view label propagation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 825-838, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.

Natasha Noy, Markus Nentwig, Michael Hartung, Axel-Cyrille Ngonga Ngomo, and Erhard Rahm. 2017. A survey of current link discovery frameworks. Semantic Web, 8(3):419-436.

Zhiyuan Qi, Ziheng Zhang, Jiaoyan Chen, Xi Chen, Yuejia Xiang, Ningyu Zhang, and Yefeng Zheng. 2021. Unsupervised knowledge graph alignment by probabilistic reasoning and semantic embedding. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pages 20192025. International Joint Conferences on Artificial Intelligence Organization. Main Track.

Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. Zero: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1-16. IEEE.

Thomas Rebele, Fabian M. Suchanek, Johannes Hoffart, Joanna Asia Biega, Erdal Kuzey, and Gerhard Weikum. 2016. Yago: A multilingual knowledge base from wikipedia, wordnet, and geonames. In Proceedings of the Fifteenth International Semantic Web Conference (ISWC2016), pages 177-185.

Fabian M. Suchanek, Serge Abiteboul, and Pierre Senellart. 2011. Paris: probabilistic alignment of relations, instances, and schema. Proc. VLDB Endow., 5(3):157-168.

Zequn Sun, Wei Hu, and Chengkai Li. 2017. Cross-lingual entity alignment via joint attribute-preserving embedding. In The Semantic Web-ISWC 2017: 16th International Semantic Web Conference, Vienna, Austria, October 21-25, 2017, Proceedings, Part I 16, pages 628-644. Springer.

Zequn Sun, Wei Hu, Qingheng Zhang, and Yuzhong Qu. 2018. Bootstrapping entity alignment with knowledge graph embedding. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pages 4396-4402. International Joint Conferences on Artificial Intelligence Organization.

Zequn Sun, Chengming Wang, Wei Hu, Muhao Chen, Jian Dai, Wei Zhang, and Yuzhong Qu. 2020a. Knowledge graph alignment network with gated multi-hop neighborhood aggregation. Proceedings of the AAAI Conference on Artificial Intelligence, 34(01):222-229.

Zequn Sun, Qingheng Zhang, Wei Hu, Chengming Wang, Muhao Chen, Farahnaz Akrami, and Chengkai Li. 2020b. A benchmarking study of embedding-based entity alignment for knowledge graphs. Proceedings of the VLDB Endowment, 13:2326 - 2340.

Jianheng Tang, Kangfei Zhao, and Jia Li. 2023. A fused Gromov-Wasserstein framework for unsupervised knowledge graph entity alignment. In Findings of the Association for Computational Linguistics: ACL 2023, pages 3320-3334, Toronto, Canada. Association for Computational Linguistics.

Xiaobin Tang, Jing Zhang, Bo Chen, Yang Yang, Hong Chen, and Cuiping Li. 2020. BERTINT:a bert-based interaction model for knowledge graph alignment. In Proceedings of the

Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pages 3174-3180. International Joint Conferences on Artificial Intelligence Organization. Main track.

Denny Vrandeˇ ci´ c and Markus Krötzsch. 2014. Wikidata: a free collaborative knowledgebase. Commun. ACM, 57(10):78-85.

Zhichun Wang, Qingsong Lv, Xiaohan Lan, and Yu Zhang. 2018. Cross-lingual knowledge graph alignment via graph convolutional networks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 349-357, Brussels, Belgium. Association for Computational Linguistics.

Yuting Wu, Xiao Liu, Yansong Feng, Zheng Wang, Rui Yan, and Dongyan Zhao. 2019a. Relation-aware entity alignment for heterogeneous knowledge graphs. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pages 5278-5284. International Joint Conferences on Artificial Intelligence Organization.

Yuting Wu, Xiao Liu, Yansong Feng, Zheng Wang, and Dongyan Zhao. 2019b. Jointly learning entity and relation representations for entity alignment. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 240-249, Hong Kong, China. Association for Computational Linguistics.

Yuting Wu, Xiao Liu, Yansong Feng, Zheng Wang, and Dongyan Zhao. 2020. Neighborhood matching network for entity alignment. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6477-6487, Online. Association for Computational Linguistics.

Kun Xu, Liwei Wang, Mo Yu, Yansong Feng, Yan Song, Zhiguo Wang, and Dong Yu. 2019. Cross-lingual knowledge graph alignment via graph matching neural network. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 31563161, Florence, Italy. Association for Computational Linguistics.

Hsiu-Wei Yang, Yanyan Zou, Peng Shi, Wei Lu, Jimmy Lin, and Xu Sun. 2019. Aligning cross-lingual entities with multi-aspect information. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4431-4441, Hong Kong, China. Association for Computational Linguistics.

Linyao Yang, Hongyang Chen, Xiao Wang, Jing Yang, Fei-Yue Wang, and Han Liu. 2024. Two heads are better than one: Integrating knowledge from knowledge graphs and large language models for entity alignment.

Kaisheng Zeng, Zhenhao Dong, Lei Hou, Yixin Cao, Minghao Hu, Jifan Yu, Xin Lv, Lei Cao, Xin Wang, Haozhuang Liu, Yi Huang, Junlan Feng, Jing Wan, Juanzi Li, and Ling Feng. 2022. Interactive contrastive learning for selfsupervised entity alignment. In Proceedings of the 31st ACM International Conference on Information &amp; Knowledge Management, CIKM '22, pages 2465-2475, New York, NY, USA. Association for Computing Machinery.

Weixin Zeng, Xiang Zhao, Jiuyang Tang, and Xuemin Lin. 2020. Collective entity alignment via adaptive features. In 2020 IEEE 36th International Conference on Data Engineering (ICDE), pages 1870-1873.

Qingheng Zhang, Zequn Sun, Wei Hu, Muhao Chen, Lingbing Guo, and Yuzhong Qu. 2019. Multi-view knowledge graph embedding for entity alignment. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pages 5429-5435. International Joint Conferences on Artificial Intelligence Organization.

Rui Zhang, Yixin Su, Bayu Distiawan Trisedya, Xiaoyan Zhao, Min Yang, Hong Cheng, and Jianzhong Qi. 2023. Autoalign: Fully automatic and effective knowledge graph alignment enabled by large language models. IEEE Transactions on Knowledge and Data Engineering, pages 1-14.

Ziheng Zhang, Hualuo Liu, Jiaoyan Chen, Xi Chen, Bo Liu, YueJia Xiang, and

Yefeng Zheng. 2020. An industry evaluation of embedding-based entity alignment. In Proceedings of the 28th International Conference on Computational Linguistics: Industry Track, pages 179-189, Online. International Committee on Computational Linguistics.

Yu Zhao, Yike Wu, Xiangrui Cai, Ying Zhang, Haiwei Zhang, and Xiaojie Yuan. 2023. From alignment to entailment: A unified textual entailment framework for entity alignment. In Findings of the Association for Computational Linguistics: ACL 2023, pages 8795-8806, Toronto, Canada. Association for Computational Linguistics.

Ziyue Zhong, Meihui Zhang, Ju Fan, and Chenxiao Dou. 2022. Semantics driven embedding learning for effective entity alignment. In 2022 IEEE 38th International Conference on Data Engineering (ICDE), pages 2127-2140.

Hao Zhu, Ruobing Xie, Zhiyuan Liu, and Maosong Sun. 2017. Iterative entity alignment via joint knowledge embeddings. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, pages 4258-4264.

Qiannan Zhu, Xiaofei Zhou, Jia Wu, Jianlong Tan, and Li Guo. 2019. Neighborhoodaware attentional representation for multilingual knowledge graphs. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pages 1943-1949. International Joint Conferences on Artificial Intelligence Organization.
Zhichun Wang Xuan Chen
DERA: Dense Entity Retrieval for Entity Alignment in Knowledge Graphs
Entity Alignment (EA) aims to match equivalent entities in different
Knowledge Graphs (KGs), which is essential for knowledge fusion and
integration. Recently, embedding-based EA has attracted significant attention
and many approaches have been proposed. Early approaches primarily focus on
learning entity embeddings from the structural features of KGs, defined by
relation triples. Later methods incorporated entities' names and attributes as
auxiliary information to enhance embeddings for EA. However, these approaches
often used different techniques to encode structural and attribute information,
limiting their interaction and mutual enhancement. In this work, we propose a
dense entity retrieval framework for EA, leveraging language models to
uniformly encode various features of entities and facilitate nearest entity
search across KGs. Alignment candidates are first generated through entity
retrieval, which are subsequently reranked to determine the final alignments.
We conduct comprehensive experiments on both cross-lingual and monolingual EA
datasets, demonstrating that our approach achieves state-of-the-art performance
compared to existing EA methods.