Self-supervised Quantized Representation for Seamlessly Integrating
  Knowledge Graphs with Large Language Models
Due to the presence of the natural gap between Knowledge Graph (KG)
structures and the natural language, the effective integration of holistic
structural information of KGs with Large Language Models (LLMs) has emerged as
a significant question. To this end, we propose a two-stage framework to learn
and apply quantized codes for each entity, aiming for the seamless integration
of KGs with LLMs. Firstly, a self-supervised quantized representation (SSQR)
method is proposed to compress both KG structural and semantic knowledge into
discrete codes (\ie, tokens) that align the format of language sentences. We
further design KG instruction-following data by viewing these learned codes as
features to directly input to LLMs, thereby achieving seamless integration. The
experiment results demonstrate that SSQR outperforms existing unsupervised
quantized methods, producing more distinguishable codes. Further, the
fine-tuned LLaMA2 and LLaMA3.1 also have superior performance on KG link
prediction and triple classification tasks, utilizing only 16 tokens per entity
instead of thousands in conventional prompting methods.
67a0a3201d9fadf4470cb07a
Qika Lin
extracted_pending
2025-02-03T11:06:10.149Z
66ac77011cfb12c087605acb
/avatars/54c06bd1c4c9d491470ed4162c2301ae.svg
Lin
user
Qika
67a0a3201d9fadf4470cb07b
Tianzhe Zhao
67a0a3201d9fadf4470cb07c
Kai He
67a0a3201d9fadf4470cb07d
Zhen Peng
67a0a3201d9fadf4470cb07e
Fangzhi Xu
claimed_verified
2025-03-20T10:54:22.151Z
64e6cf78ecce34cb442dc889
https://cdn-avatars.huggingface.co/v1/production/uploads/64e6cf78ecce34cb442dc889/qVZFiUEpBpSkmH8SQeinm.jpeg
Fangzhi Xu
user
xufangzhi
67a0a3201d9fadf4470cb07f
Ling Huang
67a0a3201d9fadf4470cb080
Jingying Ma
67a0a3201d9fadf4470cb081
Mengling Feng
Self-supervised Quantized Representation for Seamlessly Integrating
  Knowledge Graphs with Large Language Models
Due to the presence of the natural gap between Knowledge Graph (KG)
structures and the natural language, the effective integration of holistic
structural information of KGs with Large Language Models (LLMs) has emerged as
a significant question. To this end, we propose a two-stage framework to learn
and apply quantized codes for each entity, aiming for the seamless integration
of KGs with LLMs. Firstly, a self-supervised quantized representation (SSQR)
method is proposed to compress both KG structural and semantic knowledge into
discrete codes (\ie, tokens) that align the format of language sentences. We
further design KG instruction-following data by viewing these learned codes as
features to directly input to LLMs, thereby achieving seamless integration. The
experiment results demonstrate that SSQR outperforms existing unsupervised
quantized methods, producing more distinguishable codes. Further, the
fine-tuned LLaMA2 and LLaMA3.1 also have superior performance on KG link
prediction and triple classification tasks, utilizing only 16 tokens per entity
instead of thousands in conventional prompting methods.
A two-stage framework using self-supervised quantized representation (SSQR) integrates Knowledge Graphs (KGs) with Large Language Models (LLMs), producing more distinguishable codes and enhancing performance on KG tasks.
Knowledge Graph (KG) structural information Large Language Models (LLMs) self-supervised quantized representation (SSQR) discrete codes KG instruction-following data fine-tuned LLaMA2 LLaMA3.1 KG link prediction triple classification