title: Grounding Visual Illusions in Language: Do Vision-Language Models
    Perceive Illusions Like Humans?
summary: Vision-Language Models (VLMs) are trained on vast amounts of data captured by
  humans emulating our understanding of the world. However, known as visual
  illusions, human's perception of reality isn't always faithful to the physical
  world. This raises a key question: do VLMs have the similar kind of illusions
  as humans do, or do they faithfully learn to represent reality? To investigate
  this question, we build a dataset containing five types of visual illusions and
  formulate four tasks to examine visual illusions in state-of-the-art VLMs. Our
  findings have shown that although the overall alignment is low, larger models
  are closer to human perception and more susceptible to visual illusions. Our
  dataset and initial findings will promote a better understanding of visual
  illusions in humans and machines and provide a stepping stone for future
  computational models that can better align humans and machines in perceiving
  and communicating about the shared visual world. The code and data are
  available at https://github.com/vl-illusion/dataset.
paper_authors._id: 65430c2222af8fe4f52db823
paper_authors.name: Yichi Zhang
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-02-28T06:45:27.897Z
paper_authors._id: 6369b1d456d1f93498130a8a
paper_authors.avatarUrl: /avatars/8ec228aa6f171715652511f948765db9.svg
paper_authors.fullname: Yichi Zhang
paper_authors.type: user
paper_authors.user: 594zyc
paper_authors._id: 65430c2222af8fe4f52db824
paper_authors.name: Jiayi Pan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-02T13:48:48.368Z
paper_authors._id: 61568f37272f2d87a99ba884
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61568f37272f2d87a99ba884/lgvkl5f0rEyiQRVU5FE32.png
paper_authors.fullname: Jiayi Pan
paper_authors.type: user
paper_authors.user: Jiayi-Pan
paper_authors._id: 65430c2222af8fe4f52db825
paper_authors.name: Yuchen Zhou
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-03-25T05:15:42.469Z
paper_authors._id: 660108157a3a9bbfcd888729
paper_authors.avatarUrl: /avatars/f1b0b616e8bf0912ff55aceefd41ba28.svg
paper_authors.fullname: Anna Zhou
paper_authors.type: user
paper_authors.user: ehaemmma
paper_authors._id: 65430c2222af8fe4f52db826
paper_authors.name: Rui Pan
paper_authors._id: 65430c2222af8fe4f52db827
paper_authors.name: Joyce Chai
paper_title: Grounding Visual Illusions in Language: Do Vision-Language Models
    Perceive Illusions Like Humans?
paper_summary: Vision-Language Models (VLMs) are trained on vast amounts of data captured by
  humans emulating our understanding of the world. However, known as visual
  illusions, human's perception of reality isn't always faithful to the physical
  world. This raises a key question: do VLMs have the similar kind of illusions
  as humans do, or do they faithfully learn to represent reality? To investigate
  this question, we build a dataset containing five types of visual illusions and
  formulate four tasks to examine visual illusions in state-of-the-art VLMs. Our
  findings have shown that although the overall alignment is low, larger models
  are closer to human perception and more susceptible to visual illusions. Our
  dataset and initial findings will promote a better understanding of visual
  illusions in humans and machines and provide a stepping stone for future
  computational models that can better align humans and machines in perceiving
  and communicating about the shared visual world. The code and data are
  available at https://github.com/vl-illusion/dataset.
paper_ai_summary: Visual-Language Models exhibit susceptibility to visual illusions, similar to humans, with larger models showing higher alignment and susceptibility compared to smaller models.
paper_ai_keywords: vision-language models visual illusions dataset human perception