title: Deliberation on Priors: Trustworthy Reasoning of Large Language Models
    on Knowledge Graphs
summary: Knowledge graph-based retrieval-augmented generation seeks to mitigate
  hallucinations in Large Language Models (LLMs) caused by insufficient or
  outdated knowledge. However, existing methods often fail to fully exploit the
  prior knowledge embedded in knowledge graphs (KGs), particularly their
  structural information and explicit or implicit constraints. The former can
  enhance the faithfulness of LLMs' reasoning, while the latter can improve the
  reliability of response generation. Motivated by these, we propose a
  trustworthy reasoning framework, termed Deliberation over Priors (DP), which
  sufficiently utilizes the priors contained in KGs. Specifically, DP adopts a
  progressive knowledge distillation strategy that integrates structural priors
  into LLMs through a combination of supervised fine-tuning and Kahneman-Tversky
  optimization, thereby improving the faithfulness of relation path generation.
  Furthermore, our framework employs a reasoning-introspection strategy, which
  guides LLMs to perform refined reasoning verification based on extracted
  constraint priors, ensuring the reliability of response generation. Extensive
  experiments on three benchmark datasets demonstrate that DP achieves new
  state-of-the-art performance, especially a Hit@1 improvement of 13% on the
  ComplexWebQuestions dataset, and generates highly trustworthy responses. We
  also conduct various analyses to verify its flexibility and practicality. The
  code is available at https://github.com/reml-group/Deliberation-on-Priors.
paper_authors._id: 682e84d83291b134b3370184
paper_authors.name: Jie Ma
paper_authors._id: 682e84d83291b134b3370185
paper_authors.name: Ning Qu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-22T07:16:55.205Z
paper_authors._id: 653d0136d6f7982a7a52054d
paper_authors.avatarUrl: /avatars/5d47d022dd5954a4c3d243a7f0292d32.svg
paper_authors.fullname: QUNING
paper_authors.type: user
paper_authors.user: stillqu
paper_authors._id: 682e84d83291b134b3370186
paper_authors.name: Zhitao Gao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-26T08:22:25.806Z
paper_authors._id: 673c80505398ab701feb8c50
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/OQ-4gDCZ61eJ8ERX72tnH.png
paper_authors.fullname: ZhitaoGao
paper_authors.type: user
paper_authors.user: ZhitaoGao
paper_authors._id: 682e84d83291b134b3370187
paper_authors.name: Rui Xing
paper_authors._id: 682e84d83291b134b3370188
paper_authors.name: Jun Liu
paper_authors._id: 682e84d83291b134b3370189
paper_authors.name: Hongbin Pei
paper_authors._id: 682e84d83291b134b337018a
paper_authors.name: Jiang Xie
paper_authors._id: 682e84d83291b134b337018b
paper_authors.name: Linyun Song
paper_authors._id: 682e84d83291b134b337018c
paper_authors.name: Pinghui Wang
paper_authors._id: 682e84d83291b134b337018d
paper_authors.name: Jing Tao
paper_authors._id: 682e84d83291b134b337018e
paper_authors.name: Zhou Su
paper_title: Deliberation on Priors: Trustworthy Reasoning of Large Language Models
    on Knowledge Graphs
paper_summary: Knowledge graph-based retrieval-augmented generation seeks to mitigate
  hallucinations in Large Language Models (LLMs) caused by insufficient or
  outdated knowledge. However, existing methods often fail to fully exploit the
  prior knowledge embedded in knowledge graphs (KGs), particularly their
  structural information and explicit or implicit constraints. The former can
  enhance the faithfulness of LLMs' reasoning, while the latter can improve the
  reliability of response generation. Motivated by these, we propose a
  trustworthy reasoning framework, termed Deliberation over Priors (DP), which
  sufficiently utilizes the priors contained in KGs. Specifically, DP adopts a
  progressive knowledge distillation strategy that integrates structural priors
  into LLMs through a combination of supervised fine-tuning and Kahneman-Tversky
  optimization, thereby improving the faithfulness of relation path generation.
  Furthermore, our framework employs a reasoning-introspection strategy, which
  guides LLMs to perform refined reasoning verification based on extracted
  constraint priors, ensuring the reliability of response generation. Extensive
  experiments on three benchmark datasets demonstrate that DP achieves new
  state-of-the-art performance, especially a Hit@1 improvement of 13% on the
  ComplexWebQuestions dataset, and generates highly trustworthy responses. We
  also conduct various analyses to verify its flexibility and practicality. The
  code is available at https://github.com/reml-group/Deliberation-on-Priors.
paper_ai_summary: The Deliberation over Priors framework enhances the trustworthiness of LLMs by integrating structural and constraint priors from knowledge graphs through knowledge distillation and reasoning introspection.
paper_ai_keywords: knowledge graph-based retrieval-augmented generation Large Language Models (LLMs) knowledge graphs (KGs) structural information explicit constraints implicit constraints trustworthworthy reasoning framework Deliberation over Priors (DP) progressive knowledge distillation supervised fine-tuning Kahneman-Tversky optimization relation path generation reasoning-introspection strategy ComplexWebQuestions dataset Hit@1 improvement