Grounding Visual Illusions in Language: Do Vision-Language Models
  Perceive Illusions Like Humans?
Vision-Language Models (VLMs) are trained on vast amounts of data captured by
humans emulating our understanding of the world. However, known as visual
illusions, human's perception of reality isn't always faithful to the physical
world. This raises a key question: do VLMs have the similar kind of illusions
as humans do, or do they faithfully learn to represent reality? To investigate
this question, we build a dataset containing five types of visual illusions and
formulate four tasks to examine visual illusions in state-of-the-art VLMs. Our
findings have shown that although the overall alignment is low, larger models
are closer to human perception and more susceptible to visual illusions. Our
dataset and initial findings will promote a better understanding of visual
illusions in humans and machines and provide a stepping stone for future
computational models that can better align humans and machines in perceiving
and communicating about the shared visual world. The code and data are
available at https://github.com/vl-illusion/dataset.
65430c2222af8fe4f52db823
Yichi Zhang
extracted_confirmed
2024-02-28T06:45:27.897Z
6369b1d456d1f93498130a8a
/avatars/8ec228aa6f171715652511f948765db9.svg
Yichi Zhang
user
594zyc
65430c2222af8fe4f52db824
Jiayi Pan
admin_assigned
2023-11-02T13:48:48.368Z
61568f37272f2d87a99ba884
https://cdn-avatars.huggingface.co/v1/production/uploads/61568f37272f2d87a99ba884/lgvkl5f0rEyiQRVU5FE32.png
Jiayi Pan
user
Jiayi-Pan
65430c2222af8fe4f52db825
Yuchen Zhou
extracted_pending
2024-03-25T05:15:42.469Z
660108157a3a9bbfcd888729
/avatars/f1b0b616e8bf0912ff55aceefd41ba28.svg
Anna Zhou
user
ehaemmma
65430c2222af8fe4f52db826
Rui Pan
65430c2222af8fe4f52db827
Joyce Chai
Grounding Visual Illusions in Language: Do Vision-Language Models
  Perceive Illusions Like Humans?
Vision-Language Models (VLMs) are trained on vast amounts of data captured by
humans emulating our understanding of the world. However, known as visual
illusions, human's perception of reality isn't always faithful to the physical
world. This raises a key question: do VLMs have the similar kind of illusions
as humans do, or do they faithfully learn to represent reality? To investigate
this question, we build a dataset containing five types of visual illusions and
formulate four tasks to examine visual illusions in state-of-the-art VLMs. Our
findings have shown that although the overall alignment is low, larger models
are closer to human perception and more susceptible to visual illusions. Our
dataset and initial findings will promote a better understanding of visual
illusions in humans and machines and provide a stepping stone for future
computational models that can better align humans and machines in perceiving
and communicating about the shared visual world. The code and data are
available at https://github.com/vl-illusion/dataset.
Visual-Language Models exhibit susceptibility to visual illusions, similar to humans, with larger models showing higher alignment and susceptibility compared to smaller models.
vision-language models visual illusions dataset human perception