title: Self-supervised Quantized Representation for Seamlessly Integrating
    Knowledge Graphs with Large Language Models
summary: Due to the presence of the natural gap between Knowledge Graph (KG)
  structures and the natural language, the effective integration of holistic
  structural information of KGs with Large Language Models (LLMs) has emerged as
  a significant question. To this end, we propose a two-stage framework to learn
  and apply quantized codes for each entity, aiming for the seamless integration
  of KGs with LLMs. Firstly, a self-supervised quantized representation (SSQR)
  method is proposed to compress both KG structural and semantic knowledge into
  discrete codes (\ie, tokens) that align the format of language sentences. We
  further design KG instruction-following data by viewing these learned codes as
  features to directly input to LLMs, thereby achieving seamless integration. The
  experiment results demonstrate that SSQR outperforms existing unsupervised
  quantized methods, producing more distinguishable codes. Further, the
  fine-tuned LLaMA2 and LLaMA3.1 also have superior performance on KG link
  prediction and triple classification tasks, utilizing only 16 tokens per entity
  instead of thousands in conventional prompting methods.
paper_authors._id: 67a0a3201d9fadf4470cb07a
paper_authors.name: Qika Lin
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-02-03T11:06:10.149Z
paper_authors._id: 66ac77011cfb12c087605acb
paper_authors.avatarUrl: /avatars/54c06bd1c4c9d491470ed4162c2301ae.svg
paper_authors.fullname: Lin
paper_authors.type: user
paper_authors.user: Qika
paper_authors._id: 67a0a3201d9fadf4470cb07b
paper_authors.name: Tianzhe Zhao
paper_authors._id: 67a0a3201d9fadf4470cb07c
paper_authors.name: Kai He
paper_authors._id: 67a0a3201d9fadf4470cb07d
paper_authors.name: Zhen Peng
paper_authors._id: 67a0a3201d9fadf4470cb07e
paper_authors.name: Fangzhi Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-20T10:54:22.151Z
paper_authors._id: 64e6cf78ecce34cb442dc889
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64e6cf78ecce34cb442dc889/qVZFiUEpBpSkmH8SQeinm.jpeg
paper_authors.fullname: Fangzhi Xu
paper_authors.type: user
paper_authors.user: xufangzhi
paper_authors._id: 67a0a3201d9fadf4470cb07f
paper_authors.name: Ling Huang
paper_authors._id: 67a0a3201d9fadf4470cb080
paper_authors.name: Jingying Ma
paper_authors._id: 67a0a3201d9fadf4470cb081
paper_authors.name: Mengling Feng
paper_title: Self-supervised Quantized Representation for Seamlessly Integrating
    Knowledge Graphs with Large Language Models
paper_summary: Due to the presence of the natural gap between Knowledge Graph (KG)
  structures and the natural language, the effective integration of holistic
  structural information of KGs with Large Language Models (LLMs) has emerged as
  a significant question. To this end, we propose a two-stage framework to learn
  and apply quantized codes for each entity, aiming for the seamless integration
  of KGs with LLMs. Firstly, a self-supervised quantized representation (SSQR)
  method is proposed to compress both KG structural and semantic knowledge into
  discrete codes (\ie, tokens) that align the format of language sentences. We
  further design KG instruction-following data by viewing these learned codes as
  features to directly input to LLMs, thereby achieving seamless integration. The
  experiment results demonstrate that SSQR outperforms existing unsupervised
  quantized methods, producing more distinguishable codes. Further, the
  fine-tuned LLaMA2 and LLaMA3.1 also have superior performance on KG link
  prediction and triple classification tasks, utilizing only 16 tokens per entity
  instead of thousands in conventional prompting methods.
paper_ai_summary: A two-stage framework using self-supervised quantized representation (SSQR) integrates Knowledge Graphs (KGs) with Large Language Models (LLMs), producing more distinguishable codes and enhancing performance on KG tasks.
paper_ai_keywords: Knowledge Graph (KG) structural information Large Language Models (LLMs) self-supervised quantized representation (SSQR) discrete codes KG instruction-following data fine-tuned LLaMA2 LLaMA3.1 KG link prediction triple classification