title: LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond
summary: With the recent appearance of LLMs in practical settings, having methods that
  can effectively detect factual inconsistencies is crucial to reduce the
  propagation of misinformation and improve trust in model outputs. When testing
  on existing factual consistency benchmarks, we find that a few large language
  models (LLMs) perform competitively on classification benchmarks for factual
  inconsistency detection compared to traditional non-LLM methods. However, a
  closer analysis reveals that most LLMs fail on more complex formulations of the
  task and exposes issues with existing evaluation benchmarks, affecting
  evaluation precision. To address this, we propose a new protocol for
  inconsistency detection benchmark creation and implement it in a 10-domain
  benchmark called SummEdits. This new benchmark is 20 times more cost-effective
  per sample than previous benchmarks and highly reproducible, as we estimate
  inter-annotator agreement at about 0.9. Most LLMs struggle on SummEdits, with
  performance close to random chance. The best-performing model, GPT-4, is still
  8\% below estimated human performance, highlighting the gaps in LLMs' ability
  to reason about facts and detect inconsistencies when they occur.
paper_authors._id: 646edd6566f7b97a94feb3c1
paper_authors.name: Philippe Laban
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-25T15:20:57.688Z
paper_authors._id: 610aee2dc30263ce69e0dd9c
paper_authors.avatarUrl: /avatars/b6e5bba78425e9c28d1b75ee8a7eaac1.svg
paper_authors.fullname: Philippe Laban
paper_authors.type: user
paper_authors.user: philippelaban
paper_authors._id: 646edd6566f7b97a94feb3c2
paper_authors.name: Wojciech Kryściński
paper_authors._id: 646edd6566f7b97a94feb3c3
paper_authors.name: Divyansh Agarwal
paper_authors._id: 646edd6566f7b97a94feb3c4
paper_authors.name: Alexander R. Fabbri
paper_authors._id: 646edd6566f7b97a94feb3c5
paper_authors.name: Caiming Xiong
paper_authors._id: 646edd6566f7b97a94feb3c6
paper_authors.name: Shafiq Joty
paper_authors._id: 646edd6566f7b97a94feb3c7
paper_authors.name: Chien-Sheng Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-26T08:07:29.023Z
paper_authors._id: 5f09073719cb630495b81428
paper_authors.avatarUrl: /avatars/e49f53128b5305dfec44d80129c59ca3.svg
paper_authors.fullname: Chien-Sheng (Jason) Wu
paper_authors.type: user
paper_authors.user: jasonwu
paper_title: LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond
paper_summary: With the recent appearance of LLMs in practical settings, having methods that
  can effectively detect factual inconsistencies is crucial to reduce the
  propagation of misinformation and improve trust in model outputs. When testing
  on existing factual consistency benchmarks, we find that a few large language
  models (LLMs) perform competitively on classification benchmarks for factual
  inconsistency detection compared to traditional non-LLM methods. However, a
  closer analysis reveals that most LLMs fail on more complex formulations of the
  task and exposes issues with existing evaluation benchmarks, affecting
  evaluation precision. To address this, we propose a new protocol for
  inconsistency detection benchmark creation and implement it in a 10-domain
  benchmark called SummEdits. This new benchmark is 20 times more cost-effective
  per sample than previous benchmarks and highly reproducible, as we estimate
  inter-annotator agreement at about 0.9. Most LLMs struggle on SummEdits, with
  performance close to random chance. The best-performing model, GPT-4, is still
  8\% below estimated human performance, highlighting the gaps in LLMs' ability
  to reason about facts and detect inconsistencies when they occur.
paper_ai_summary: A new benchmark called SummEdits shows that most large language models struggle to detect factual inconsistencies effectively, highlighting gaps in their reasoning abilities.
paper_ai_keywords: LLMs factual consistency benchmarks SummEdits inter-annotator agreement