title: Can Knowledge Editing Really Correct Hallucinations?
summary: Large Language Models (LLMs) suffer from hallucinations, referring to the
  non-factual information in generated content, despite their superior capacities
  across tasks. Meanwhile, knowledge editing has been developed as a new popular
  paradigm to correct the erroneous factual knowledge encoded in LLMs with the
  advantage of avoiding retraining from scratch. However, one common issue of
  existing evaluation datasets for knowledge editing is that they do not ensure
  LLMs actually generate hallucinated answers to the evaluation questions before
  editing. When LLMs are evaluated on such datasets after being edited by
  different techniques, it is hard to directly adopt the performance to assess
  the effectiveness of different knowledge editing methods in correcting
  hallucinations. Thus, the fundamental question remains insufficiently
  validated: Can knowledge editing really correct hallucinations in LLMs? We
  proposed HalluEditBench to holistically benchmark knowledge editing methods in
  correcting real-world hallucinations. First, we rigorously construct a massive
  hallucination dataset with 9 domains, 26 topics and more than 6,000
  hallucinations. Then, we assess the performance of knowledge editing methods in
  a holistic way on five dimensions including Efficacy, Generalization,
  Portability, Locality, and Robustness. Through HalluEditBench, we have provided
  new insights into the potentials and limitations of different knowledge editing
  methods in correcting hallucinations, which could inspire future improvements
  and facilitate the progress in the field of knowledge editing.
paper_authors._id: 67171d2869532dd147c9d9f8
paper_authors.name: Baixiang Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-25T11:35:51.119Z
paper_authors._id: 64821657564f4a586629a76a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64821657564f4a586629a76a/LoBNDQFD7yHiueESY14Ic.jpeg
paper_authors.fullname: Baixiang Huang
paper_authors.type: user
paper_authors.user: BaixHuang
paper_authors._id: 67171d2869532dd147c9d9f9
paper_authors.name: Canyu Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-25T09:31:07.681Z
paper_authors._id: 6483af58571c2dcfa98cae82
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6483af58571c2dcfa98cae82/d8Vnh_EG1KxeEoNAAQXHZ.jpeg
paper_authors.fullname: Canyu Chen
paper_authors.type: user
paper_authors.user: canyuchen
paper_authors._id: 67171d2869532dd147c9d9fa
paper_authors.name: Xiongxiao Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-25T11:35:57.623Z
paper_authors._id: 65da7f82ad7ccf910d246371
paper_authors.avatarUrl: /avatars/d28b12d703d3bc62ac4b8ea30faae7c9.svg
paper_authors.fullname: Xiongxiao Xu
paper_authors.type: user
paper_authors.user: XiongxiaoXu
paper_authors._id: 67171d2869532dd147c9d9fb
paper_authors.name: Ali Payani
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-25T11:36:04.198Z
paper_authors._id: 6411f7af46e5f8976912d62a
paper_authors.avatarUrl: /avatars/d8d5b012dc7704619fcd37b809b43059.svg
paper_authors.fullname: Ali Payani
paper_authors.type: user
paper_authors.user: apayani
paper_authors._id: 67171d2869532dd147c9d9fc
paper_authors.name: Kai Shu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-25T11:36:10.091Z
paper_authors._id: 671774e5fc12736836da240f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/dE_--2tAQaEP7atMh35Ed.png
paper_authors.fullname: Kai Shu
paper_authors.type: user
paper_authors.user: kaishu666
paper_title: Can Knowledge Editing Really Correct Hallucinations?
paper_summary: Large Language Models (LLMs) suffer from hallucinations, referring to the
  non-factual information in generated content, despite their superior capacities
  across tasks. Meanwhile, knowledge editing has been developed as a new popular
  paradigm to correct the erroneous factual knowledge encoded in LLMs with the
  advantage of avoiding retraining from scratch. However, one common issue of
  existing evaluation datasets for knowledge editing is that they do not ensure
  LLMs actually generate hallucinated answers to the evaluation questions before
  editing. When LLMs are evaluated on such datasets after being edited by
  different techniques, it is hard to directly adopt the performance to assess
  the effectiveness of different knowledge editing methods in correcting
  hallucinations. Thus, the fundamental question remains insufficiently
  validated: Can knowledge editing really correct hallucinations in LLMs? We
  proposed HalluEditBench to holistically benchmark knowledge editing methods in
  correcting real-world hallucinations. First, we rigorously construct a massive
  hallucination dataset with 9 domains, 26 topics and more than 6,000
  hallucinations. Then, we assess the performance of knowledge editing methods in
  a holistic way on five dimensions including Efficacy, Generalization,
  Portability, Locality, and Robustness. Through HalluEditBench, we have provided
  new insights into the potentials and limitations of different knowledge editing
  methods in correcting hallucinations, which could inspire future improvements
  and facilitate the progress in the field of knowledge editing.
paper_ai_keywords: hallucinations knowledge editing retraining from scratch evaluation datasets factual knowledge LLMs (Language Models) generated hallucinations evaluation questions real-world hallucinations HalluEditBench hallucination dataset performance Efficacy Generalization Portability Locality Robustness knowledge editing methods