title: ConflictBank: A Benchmark for Evaluating the Influence of Knowledge
    Conflicts in LLM
summary: Large language models (LLMs) have achieved impressive advancements across
  numerous disciplines, yet the critical issue of knowledge conflicts, a major
  source of hallucinations, has rarely been studied. Only a few research explored
  the conflicts between the inherent knowledge of LLMs and the retrieved
  contextual knowledge. However, a thorough assessment of knowledge conflict in
  LLMs is still missing. Motivated by this research gap, we present ConflictBank,
  the first comprehensive benchmark developed to systematically evaluate
  knowledge conflicts from three aspects: (i) conflicts encountered in retrieved
  knowledge, (ii) conflicts within the models' encoded knowledge, and (iii) the
  interplay between these conflict forms. Our investigation delves into four
  model families and twelve LLM instances, meticulously analyzing conflicts
  stemming from misinformation, temporal discrepancies, and semantic divergences.
  Based on our proposed novel construction framework, we create 7,453,853
  claim-evidence pairs and 553,117 QA pairs. We present numerous findings on
  model scale, conflict causes, and conflict types. We hope our ConflictBank
  benchmark will help the community better understand model behavior in conflicts
  and develop more reliable LLMs.
paper_authors._id: 66c8481a2400073af3fbd0ae
paper_authors.name: Zhaochen Su
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-10T08:21:48.988Z
paper_authors._id: 64264095ba51f8a2136946a0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64264095ba51f8a2136946a0/FR33boVpkDXcrvGMBmprF.jpeg
paper_authors.fullname: Zhaochen Su
paper_authors.type: user
paper_authors.user: Warrieryes
paper_authors._id: 66c8481a2400073af3fbd0af
paper_authors.name: Jun Zhang
paper_authors._id: 66c8481a2400073af3fbd0b0
paper_authors.name: Xiaoye Qu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-04T08:46:39.836Z
paper_authors._id: 64cb54da1af278541d663708
paper_authors.avatarUrl: /avatars/c44507cc92bb2e83154bad31b90ce6dd.svg
paper_authors.fullname: Xiaoye Qu
paper_authors.type: user
paper_authors.user: Xiaoye08
paper_authors._id: 66c8481a2400073af3fbd0b1
paper_authors.name: Tong Zhu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-04T09:02:51.536Z
paper_authors._id: 629454301ae2138079f7ff31
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/629454301ae2138079f7ff31/rVtbF-j06gDiYzomTeVTc.jpeg
paper_authors.fullname: Tong Zhu
paper_authors.type: user
paper_authors.user: Spico
paper_authors._id: 66c8481a2400073af3fbd0b2
paper_authors.name: Yanshu Li
paper_authors._id: 66c8481a2400073af3fbd0b3
paper_authors.name: Jiashuo Sun
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-14T07:37:52.927Z
paper_authors._id: 64527f701a57e1179c1c3693
paper_authors.avatarUrl: /avatars/25b2632d7aa9ce26d5d4924ecb00c4f4.svg
paper_authors.fullname: Jiashuo Sun
paper_authors.type: user
paper_authors.user: gasolsun
paper_authors._id: 66c8481a2400073af3fbd0b4
paper_authors.name: Juntao Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-04T20:11:41.428Z
paper_authors._id: 60d1d7f94672e8c13e7c2bce
paper_authors.avatarUrl: /avatars/be3a3990492ef1c5f3ff33cc043ae166.svg
paper_authors.fullname: Juntao Li
paper_authors.type: user
paper_authors.user: ljtsuda
paper_authors._id: 66c8481a2400073af3fbd0b5
paper_authors.name: Min Zhang
paper_authors._id: 66c8481a2400073af3fbd0b6
paper_authors.name: Yu Cheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-09T16:26:33.596Z
paper_authors._id: 67017abfe4d49b157ac534d9
paper_authors.avatarUrl: /avatars/997e1b9f54b27a7728a9d4abfee4ba91.svg
paper_authors.fullname: Yu Cheng
paper_authors.type: user
paper_authors.user: ych133
paper_title: ConflictBank: A Benchmark for Evaluating the Influence of Knowledge
    Conflicts in LLM
paper_summary: Large language models (LLMs) have achieved impressive advancements across
  numerous disciplines, yet the critical issue of knowledge conflicts, a major
  source of hallucinations, has rarely been studied. Only a few research explored
  the conflicts between the inherent knowledge of LLMs and the retrieved
  contextual knowledge. However, a thorough assessment of knowledge conflict in
  LLMs is still missing. Motivated by this research gap, we present ConflictBank,
  the first comprehensive benchmark developed to systematically evaluate
  knowledge conflicts from three aspects: (i) conflicts encountered in retrieved
  knowledge, (ii) conflicts within the models' encoded knowledge, and (iii) the
  interplay between these conflict forms. Our investigation delves into four
  model families and twelve LLM instances, meticulously analyzing conflicts
  stemming from misinformation, temporal discrepancies, and semantic divergences.
  Based on our proposed novel construction framework, we create 7,453,853
  claim-evidence pairs and 553,117 QA pairs. We present numerous findings on
  model scale, conflict causes, and conflict types. We hope our ConflictBank
  benchmark will help the community better understand model behavior in conflicts
  and develop more reliable LLMs.
paper_ai_summary: ConflictBank is a benchmark for evaluating knowledge conflicts in LLMs by assessing retrieved knowledge, model-encoded knowledge, and their interplay across different model families and causes like misinformation and temporal discrepancies.
paper_ai_keywords: large language models LLMs knowledge conflicts hallucinations retrieved knowledge encoded knowledge ConflictBank misinformation temporal discrepancies claim-evidence pairs QA pairs