AUTOHALLUSION: Automatic Generation of Hallucination Benchmarks for
  Vision-Language Models
Large vision-language models (LVLMs) hallucinate: certain context cues in an
image may trigger the language module's overconfident and incorrect reasoning
on abnormal or hypothetical objects. Though a few benchmarks have been
developed to investigate LVLM hallucinations, they mainly rely on hand-crafted
corner cases whose fail patterns may hardly generalize, and finetuning on them
could undermine their validity. These motivate us to develop the first
automatic benchmark generation approach, AUTOHALLUSION, that harnesses a few
principal strategies to create diverse hallucination examples. It probes the
language modules in LVLMs for context cues and uses them to synthesize images
by: (1) adding objects abnormal to the context cues; (2) for two co-occurring
objects, keeping one and excluding the other; or (3) removing objects closely
tied to the context cues. It then generates image-based questions whose
ground-truth answers contradict the language module's prior. A model has to
overcome contextual biases and distractions to reach correct answers, while
incorrect or inconsistent answers indicate hallucinations. AUTOHALLUSION
enables us to create new benchmarks at the minimum cost and thus overcomes the
fragility of hand-crafted benchmarks. It also reveals common failure patterns
and reasons, providing key insights to detect, avoid, or control
hallucinations. Comprehensive evaluations of top-tier LVLMs, e.g.,
GPT-4V(ision), Gemini Pro Vision, Claude 3, and LLaVA-1.5, show a 97.7% and
98.7% success rate of hallucination induction on synthetic and real-world
datasets of AUTOHALLUSION, paving the way for a long battle against
hallucinations.
6679e00d400b4b8da6e472d8
Xiyang Wu
6679e00d400b4b8da6e472d9
Tianrui Guan
claimed_verified
2024-07-22T07:12:24.995Z
64baeea1d1c7b6d84883e7fb
https://cdn-avatars.huggingface.co/v1/production/uploads/64baeea1d1c7b6d84883e7fb/gYEYt5T5ee50-dL1UbEzd.jpeg
Tianrui Guan
user
rayguan
6679e00d400b4b8da6e472da
Dianqi Li
6679e00d400b4b8da6e472db
Shuaiyi Huang
6679e00d400b4b8da6e472dc
Xiaoyu Liu
6679e00d400b4b8da6e472dd
Xijun Wang
6679e00d400b4b8da6e472de
Ruiqi Xian
6679e00d400b4b8da6e472df
Abhinav Shrivastava
6679e00d400b4b8da6e472e0
Furong Huang
6679e00d400b4b8da6e472e1
Jordan Lee Boyd-Graber
6679e00d400b4b8da6e472e2
Tianyi Zhou
claimed_verified
2024-07-02T21:09:00.118Z
647f5af5b0e96764589f3b2a
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VJ4cDyjp5M3V5WmI5gPIU.jpeg
Tianyi Zhou
user
zhoutianyi
6679e00d400b4b8da6e472e3
Dinesh Manocha
AUTOHALLUSION: Automatic Generation of Hallucination Benchmarks for
  Vision-Language Models
Large vision-language models (LVLMs) hallucinate: certain context cues in an
image may trigger the language module's overconfident and incorrect reasoning
on abnormal or hypothetical objects. Though a few benchmarks have been
developed to investigate LVLM hallucinations, they mainly rely on hand-crafted
corner cases whose fail patterns may hardly generalize, and finetuning on them
could undermine their validity. These motivate us to develop the first
automatic benchmark generation approach, AUTOHALLUSION, that harnesses a few
principal strategies to create diverse hallucination examples. It probes the
language modules in LVLMs for context cues and uses them to synthesize images
by: (1) adding objects abnormal to the context cues; (2) for two co-occurring
objects, keeping one and excluding the other; or (3) removing objects closely
tied to the context cues. It then generates image-based questions whose
ground-truth answers contradict the language module's prior. A model has to
overcome contextual biases and distractions to reach correct answers, while
incorrect or inconsistent answers indicate hallucinations. AUTOHALLUSION
enables us to create new benchmarks at the minimum cost and thus overcomes the
fragility of hand-crafted benchmarks. It also reveals common failure patterns
and reasons, providing key insights to detect, avoid, or control
hallucinations. Comprehensive evaluations of top-tier LVLMs, e.g.,
GPT-4V(ision), Gemini Pro Vision, Claude 3, and LLaVA-1.5, show a 97.7% and
98.7% success rate of hallucination induction on synthetic and real-world
datasets of AUTOHALLUSION, paving the way for a long battle against
hallucinations.
AUTOHALLUSION is an automatic benchmark generation approach to create diverse hallucination examples for large vision-language models, helping to detect, avoid, or control hallucinations.
large vision-language models LVLMs hallucinations automatic benchmark generation AUTOHALLUSION principal strategies context cues ground-truth answers contextual biases distractions GPT-4V Gemini Pro Vision Claude 3 LLaVA-1.5