title: ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language
    Models
summary: Large language models (LLMs) exhibit hallucinations in long-form
  question-answering tasks across various domains and wide applications. Current
  hallucination detection and mitigation datasets are limited in domains and
  sizes, which struggle to scale due to prohibitive labor costs and insufficient
  reliability of existing hallucination annotators. To facilitate the scalable
  oversight of LLM hallucinations, this paper introduces an iterative
  self-training framework that simultaneously and progressively scales up the
  hallucination annotation dataset and improves the accuracy of the hallucination
  annotator. Based on the Expectation Maximization (EM) algorithm, in each
  iteration, the framework first applies a hallucination annotation pipeline to
  annotate a scaled dataset and then trains a more accurate hallucination
  annotator on the dataset. This new hallucination annotator is adopted in the
  hallucination annotation pipeline used for the next iteration. Extensive
  experimental results demonstrate that the finally obtained hallucination
  annotator with only 7B parameters surpasses the performance of GPT-4 and
  obtains new state-of-the-art hallucination detection results on HaluEval and
  HalluQA by zero-shot inference. Such an annotator can not only evaluate the
  hallucination levels of various LLMs on the large-scale dataset but also help
  to mitigate the hallucination of LLMs generations, with the Natural Language
  Inference (NLI) metric increasing from 25% to 37% on HaluEval.
paper_authors._id: 668cb372e7cc32ad4d3a6837
paper_authors.name: Yuzhe Gu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-09T07:42:02.824Z
paper_authors._id: 6601196cc91ba4c08ad6e270
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6601196cc91ba4c08ad6e270/X2YPNzUOQXBz5Gv-xR9LW.jpeg
paper_authors.fullname: yuzhe gu
paper_authors.type: user
paper_authors.user: vanilla1116
paper_authors._id: 668cb372e7cc32ad4d3a6838
paper_authors.name: Ziwei Ji
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-09T10:37:27.463Z
paper_authors._id: 6218d7c868ec944467c5f607
paper_authors.avatarUrl: /avatars/3d2ceb17e02967aea5ef9118d8514136.svg
paper_authors.fullname: Ziwei Ji
paper_authors.type: user
paper_authors.user: ruguodayu
paper_authors._id: 668cb372e7cc32ad4d3a6839
paper_authors.name: Wenwei Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-09T10:37:33.406Z
paper_authors._id: 64e8505321540e1da3226b54
paper_authors.avatarUrl: /avatars/18958b8406d1ce492b54c1c839f18c54.svg
paper_authors.fullname: Wenwei Zhang
paper_authors.type: user
paper_authors.user: ZwwWayne
paper_authors._id: 668cb372e7cc32ad4d3a683a
paper_authors.name: Chengqi Lyu
paper_authors._id: 668cb372e7cc32ad4d3a683b
paper_authors.name: Dahua Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-09T10:37:50.609Z
paper_authors._id: 636317ed80c1a705a6eff396
paper_authors.avatarUrl: /avatars/3db090e101b916d9256d0d3e043db71d.svg
paper_authors.fullname: Dahua Lin
paper_authors.type: user
paper_authors.user: lindahua
paper_authors._id: 668cb372e7cc32ad4d3a683c
paper_authors.name: Kai Chen
paper_title: ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language
    Models
paper_summary: Large language models (LLMs) exhibit hallucinations in long-form
  question-answering tasks across various domains and wide applications. Current
  hallucination detection and mitigation datasets are limited in domains and
  sizes, which struggle to scale due to prohibitive labor costs and insufficient
  reliability of existing hallucination annotators. To facilitate the scalable
  oversight of LLM hallucinations, this paper introduces an iterative
  self-training framework that simultaneously and progressively scales up the
  hallucination annotation dataset and improves the accuracy of the hallucination
  annotator. Based on the Expectation Maximization (EM) algorithm, in each
  iteration, the framework first applies a hallucination annotation pipeline to
  annotate a scaled dataset and then trains a more accurate hallucination
  annotator on the dataset. This new hallucination annotator is adopted in the
  hallucination annotation pipeline used for the next iteration. Extensive
  experimental results demonstrate that the finally obtained hallucination
  annotator with only 7B parameters surpasses the performance of GPT-4 and
  obtains new state-of-the-art hallucination detection results on HaluEval and
  HalluQA by zero-shot inference. Such an annotator can not only evaluate the
  hallucination levels of various LLMs on the large-scale dataset but also help
  to mitigate the hallucination of LLMs generations, with the Natural Language
  Inference (NLI) metric increasing from 25% to 37% on HaluEval.
paper_ai_summary: An iterative self-training framework enhances the detection and mitigation of hallucinations in large language models by scaling the annotation dataset and improving the annotator's accuracy.
paper_ai_keywords: large language models hallucinations self-training framework Expectation Maximization hallucination annotation pipeline GPT-4 HaluEval HalluQA Natural Language Inference