title: Chain-of-Knowledge: Integrating Knowledge Reasoning into Large Language
    Models by Learning from Knowledge Graphs
summary: Large Language Models (LLMs) have exhibited impressive proficiency in various
  natural language processing (NLP) tasks, which involve increasingly complex
  reasoning. Knowledge reasoning, a primary type of reasoning, aims at deriving
  new knowledge from existing one.While it has been widely studied in the context
  of knowledge graphs (KGs), knowledge reasoning in LLMs remains underexplored.
  In this paper, we introduce Chain-of-Knowledge, a comprehensive framework for
  knowledge reasoning, including methodologies for both dataset construction and
  model learning. For dataset construction, we create KnowReason via rule mining
  on KGs. For model learning, we observe rule overfitting induced by naive
  training. Hence, we enhance CoK with a trial-and-error mechanism that simulates
  the human process of internal knowledge exploration. We conduct extensive
  experiments with KnowReason. Our results show the effectiveness of CoK in
  refining LLMs in not only knowledge reasoning, but also general reasoning
  benchmarkms.
paper_authors._id: 66839b7801c30ad93e224f2f
paper_authors.name: Yifei Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-03T15:13:55.321Z
paper_authors._id: 643c047326f177a3e41627b6
paper_authors.avatarUrl: /avatars/ade75cebd049daf080ba80a80d516240.svg
paper_authors.fullname: Yifei Zhang
paper_authors.type: user
paper_authors.user: amstrongzyf
paper_authors._id: 66839b7801c30ad93e224f30
paper_authors.name: Xintao Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-19T22:41:16.717Z
paper_authors._id: 64c7bf2c4524c2aea7eac0b3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64c7bf2c4524c2aea7eac0b3/5ocZ69MvN4RFv86Aa7ks3.png
paper_authors.fullname: Xintao Wang
paper_authors.type: user
paper_authors.user: Neph0s
paper_authors._id: 66839b7801c30ad93e224f31
paper_authors.name: Jiaqing Liang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-03T15:14:11.988Z
paper_authors._id: 634014022c72fd4ca251fed7
paper_authors.avatarUrl: /avatars/32a85d41f7435f7facfd580d9d124c33.svg
paper_authors.fullname: liangjiaqing
paper_authors.type: user
paper_authors.user: liangjiaqing
paper_authors._id: 66839b7801c30ad93e224f32
paper_authors.name: Sirui Xia
paper_authors._id: 66839b7801c30ad93e224f33
paper_authors.name: Lida Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-03T15:14:38.897Z
paper_authors._id: 63ef522ae2319f3531891451
paper_authors.avatarUrl: /avatars/9b6ea11a1b61cd96b6ac30a86e9b5329.svg
paper_authors.fullname: chenlida
paper_authors.type: user
paper_authors.user: fogdog
paper_authors._id: 66839b7801c30ad93e224f34
paper_authors.name: Yanghua Xiao
paper_title: Chain-of-Knowledge: Integrating Knowledge Reasoning into Large Language
    Models by Learning from Knowledge Graphs
paper_summary: Large Language Models (LLMs) have exhibited impressive proficiency in various
  natural language processing (NLP) tasks, which involve increasingly complex
  reasoning. Knowledge reasoning, a primary type of reasoning, aims at deriving
  new knowledge from existing one.While it has been widely studied in the context
  of knowledge graphs (KGs), knowledge reasoning in LLMs remains underexplored.
  In this paper, we introduce Chain-of-Knowledge, a comprehensive framework for
  knowledge reasoning, including methodologies for both dataset construction and
  model learning. For dataset construction, we create KnowReason via rule mining
  on KGs. For model learning, we observe rule overfitting induced by naive
  training. Hence, we enhance CoK with a trial-and-error mechanism that simulates
  the human process of internal knowledge exploration. We conduct extensive
  experiments with KnowReason. Our results show the effectiveness of CoK in
  refining LLMs in not only knowledge reasoning, but also general reasoning
  benchmarkms.
paper_ai_keywords: Large Language Models (LLMs) natural language processing (NLP) knowledge reasoning Knowledge Graphs (KGs) chain-of-knowledge (CoK) KnowReason rule mining rule overfitting trial-and-error mechanism human process of internal knowledge exploration general reasoning stability benchmarks