title: AUTOHALLUSION: Automatic Generation of Hallucination Benchmarks for
    Vision-Language Models
summary: Large vision-language models (LVLMs) hallucinate: certain context cues in an
  image may trigger the language module's overconfident and incorrect reasoning
  on abnormal or hypothetical objects. Though a few benchmarks have been
  developed to investigate LVLM hallucinations, they mainly rely on hand-crafted
  corner cases whose fail patterns may hardly generalize, and finetuning on them
  could undermine their validity. These motivate us to develop the first
  automatic benchmark generation approach, AUTOHALLUSION, that harnesses a few
  principal strategies to create diverse hallucination examples. It probes the
  language modules in LVLMs for context cues and uses them to synthesize images
  by: (1) adding objects abnormal to the context cues; (2) for two co-occurring
  objects, keeping one and excluding the other; or (3) removing objects closely
  tied to the context cues. It then generates image-based questions whose
  ground-truth answers contradict the language module's prior. A model has to
  overcome contextual biases and distractions to reach correct answers, while
  incorrect or inconsistent answers indicate hallucinations. AUTOHALLUSION
  enables us to create new benchmarks at the minimum cost and thus overcomes the
  fragility of hand-crafted benchmarks. It also reveals common failure patterns
  and reasons, providing key insights to detect, avoid, or control
  hallucinations. Comprehensive evaluations of top-tier LVLMs, e.g.,
  GPT-4V(ision), Gemini Pro Vision, Claude 3, and LLaVA-1.5, show a 97.7% and
  98.7% success rate of hallucination induction on synthetic and real-world
  datasets of AUTOHALLUSION, paving the way for a long battle against
  hallucinations.
paper_authors._id: 6679e00d400b4b8da6e472d8
paper_authors.name: Xiyang Wu
paper_authors._id: 6679e00d400b4b8da6e472d9
paper_authors.name: Tianrui Guan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-22T07:12:24.995Z
paper_authors._id: 64baeea1d1c7b6d84883e7fb
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64baeea1d1c7b6d84883e7fb/gYEYt5T5ee50-dL1UbEzd.jpeg
paper_authors.fullname: Tianrui Guan
paper_authors.type: user
paper_authors.user: rayguan
paper_authors._id: 6679e00d400b4b8da6e472da
paper_authors.name: Dianqi Li
paper_authors._id: 6679e00d400b4b8da6e472db
paper_authors.name: Shuaiyi Huang
paper_authors._id: 6679e00d400b4b8da6e472dc
paper_authors.name: Xiaoyu Liu
paper_authors._id: 6679e00d400b4b8da6e472dd
paper_authors.name: Xijun Wang
paper_authors._id: 6679e00d400b4b8da6e472de
paper_authors.name: Ruiqi Xian
paper_authors._id: 6679e00d400b4b8da6e472df
paper_authors.name: Abhinav Shrivastava
paper_authors._id: 6679e00d400b4b8da6e472e0
paper_authors.name: Furong Huang
paper_authors._id: 6679e00d400b4b8da6e472e1
paper_authors.name: Jordan Lee Boyd-Graber
paper_authors._id: 6679e00d400b4b8da6e472e2
paper_authors.name: Tianyi Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-02T21:09:00.118Z
paper_authors._id: 647f5af5b0e96764589f3b2a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VJ4cDyjp5M3V5WmI5gPIU.jpeg
paper_authors.fullname: Tianyi Zhou
paper_authors.type: user
paper_authors.user: zhoutianyi
paper_authors._id: 6679e00d400b4b8da6e472e3
paper_authors.name: Dinesh Manocha
paper_title: AUTOHALLUSION: Automatic Generation of Hallucination Benchmarks for
    Vision-Language Models
paper_summary: Large vision-language models (LVLMs) hallucinate: certain context cues in an
  image may trigger the language module's overconfident and incorrect reasoning
  on abnormal or hypothetical objects. Though a few benchmarks have been
  developed to investigate LVLM hallucinations, they mainly rely on hand-crafted
  corner cases whose fail patterns may hardly generalize, and finetuning on them
  could undermine their validity. These motivate us to develop the first
  automatic benchmark generation approach, AUTOHALLUSION, that harnesses a few
  principal strategies to create diverse hallucination examples. It probes the
  language modules in LVLMs for context cues and uses them to synthesize images
  by: (1) adding objects abnormal to the context cues; (2) for two co-occurring
  objects, keeping one and excluding the other; or (3) removing objects closely
  tied to the context cues. It then generates image-based questions whose
  ground-truth answers contradict the language module's prior. A model has to
  overcome contextual biases and distractions to reach correct answers, while
  incorrect or inconsistent answers indicate hallucinations. AUTOHALLUSION
  enables us to create new benchmarks at the minimum cost and thus overcomes the
  fragility of hand-crafted benchmarks. It also reveals common failure patterns
  and reasons, providing key insights to detect, avoid, or control
  hallucinations. Comprehensive evaluations of top-tier LVLMs, e.g.,
  GPT-4V(ision), Gemini Pro Vision, Claude 3, and LLaVA-1.5, show a 97.7% and
  98.7% success rate of hallucination induction on synthetic and real-world
  datasets of AUTOHALLUSION, paving the way for a long battle against
  hallucinations.
paper_ai_keywords: large vision-language models (LVLMs) hallucinations context cues language module overconfident and incorrect reasoning abnormal or hypothetical objects automatic benchmark generation approach AUTOHALLUSION diverse hallucination examples synthesis images co-occurring objects ground-truth answers contextual biases hallucination induction synthetic and real-world datasets GPT-4V(ision) Gemini Pro Vision Claude 3 LLaVA-1.5