title: Chain-of-Verification Reduces Hallucination in Large Language Models
summary: Generation of plausible yet incorrect factual information, termed
  hallucination, is an unsolved issue in large language models. We study the
  ability of language models to deliberate on the responses they give in order to
  correct their mistakes. We develop the Chain-of-Verification (CoVe) method
  whereby the model first (i) drafts an initial response; then (ii) plans
  verification questions to fact-check its draft; (iii) answers those questions
  independently so the answers are not biased by other responses; and (iv)
  generates its final verified response. In experiments, we show CoVe decreases
  hallucinations across a variety of tasks, from list-based questions from
  Wikidata, closed book MultiSpanQA and longform text generation.
paper_authors._id: 650bc20187dcda6616ad4cf6
paper_authors.name: Shehzaad Dhuliawala
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-21T09:47:03.315Z
paper_authors._id: 6325db12ddb9ff68f4698292
paper_authors.avatarUrl: /avatars/88e9f57a17bc9b41ddad508e7b1d2640.svg
paper_authors.fullname: Shehzaad Dhuliawala
paper_authors.type: user
paper_authors.user: shehzaadzd
paper_authors._id: 650bc20187dcda6616ad4cf7
paper_authors.name: Mojtaba Komeili
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-27T09:49:28.487Z
paper_authors._id: 659d717f2dbf5c8fee9c1fac
paper_authors.avatarUrl: /avatars/6bea1195499a66da9b48fe0c27781ba8.svg
paper_authors.fullname: Mojtaba Komeili
paper_authors.type: user
paper_authors.user: mkomeili
paper_authors._id: 650bc20187dcda6616ad4cf8
paper_authors.name: Jing Xu
paper_authors._id: 650bc20187dcda6616ad4cf9
paper_authors.name: Roberta Raileanu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-21T09:48:20.397Z
paper_authors._id: 633e94793a17ab61de8e2b9c
paper_authors.avatarUrl: /avatars/5f2f58ddeed211393660ada6b135f0d5.svg
paper_authors.fullname: Roberta Raileanu
paper_authors.type: user
paper_authors.user: rraileanu
paper_authors._id: 650bc20187dcda6616ad4cfa
paper_authors.name: Xian Li
paper_authors._id: 650bc20187dcda6616ad4cfb
paper_authors.name: Asli Celikyilmaz
paper_authors._id: 650bc20187dcda6616ad4cfc
paper_authors.name: Jason Weston
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-21T09:49:37.891Z
paper_authors._id: 62f023a36a027498eaa2f9cc
paper_authors.avatarUrl: /avatars/8ac1c5c74d0957e3c6cc94b3a7795c37.svg
paper_authors.fullname: Jason Weston
paper_authors.type: user
paper_authors.user: spermwhale
paper_title: Chain-of-Verification Reduces Hallucination in Large Language Models
paper_summary: Generation of plausible yet incorrect factual information, termed
  hallucination, is an unsolved issue in large language models. We study the
  ability of language models to deliberate on the responses they give in order to
  correct their mistakes. We develop the Chain-of-Verification (CoVe) method
  whereby the model first (i) drafts an initial response; then (ii) plans
  verification questions to fact-check its draft; (iii) answers those questions
  independently so the answers are not biased by other responses; and (iv)
  generates its final verified response. In experiments, we show CoVe decreases
  hallucinations across a variety of tasks, from list-based questions from
  Wikidata, closed book MultiSpanQA and longform text generation.
paper_ai_keywords: Chain-of-Verification (CoVe) large language models hallucination initial response verification questions fact-checking independent answers final verified response list-based questions Wikidata closed book MultiSpanQA longform text generation