title: YesBut: A High-Quality Annotated Multimodal Dataset for evaluating
    Satire Comprehension capability of Vision-Language Models
summary: Understanding satire and humor is a challenging task for even current
  Vision-Language models. In this paper, we propose the challenging tasks of
  Satirical Image Detection (detecting whether an image is satirical),
  Understanding (generating the reason behind the image being satirical), and
  Completion (given one half of the image, selecting the other half from 2 given
  options, such that the complete image is satirical) and release a high-quality
  dataset YesBut, consisting of 2547 images, 1084 satirical and 1463
  non-satirical, containing different artistic styles, to evaluate those tasks.
  Each satirical image in the dataset depicts a normal scenario, along with a
  conflicting scenario which is funny or ironic. Despite the success of current
  Vision-Language Models on multimodal tasks such as Visual QA and Image
  Captioning, our benchmarking experiments show that such models perform poorly
  on the proposed tasks on the YesBut Dataset in Zero-Shot Settings w.r.t both
  automated as well as human evaluation. Additionally, we release a dataset of
  119 real, satirical photographs for further research. The dataset and code are
  available at https://github.com/abhi1nandy2/yesbut_dataset.
paper_authors._id: 66f18738e32a9f85b62c26d3
paper_authors.name: Abhilash Nandy
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-23T16:29:36.598Z
paper_authors._id: 5f89da6c5d083370c711f37c
paper_authors.avatarUrl: /avatars/c2f17a4a636973817fd5da2ae6dbaac3.svg
paper_authors.fullname: Abhilash Nandy
paper_authors.type: user
paper_authors.user: abhi1nandy2
paper_authors._id: 66f18738e32a9f85b62c26d4
paper_authors.name: Yash Agarwal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T16:31:28.934Z
paper_authors._id: 66054e097c41a7efbc1fa842
paper_authors.avatarUrl: /avatars/11e6c01db1f962be46a2e9122bc3066b.svg
paper_authors.fullname: Yash Agarwal
paper_authors.type: user
paper_authors.user: yashagarwal
paper_authors._id: 66f18738e32a9f85b62c26d5
paper_authors.name: Ashish Patwa
paper_authors._id: 66f18738e32a9f85b62c26d6
paper_authors.name: Millon Madhur Das
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T16:31:37.988Z
paper_authors._id: 61a1284f5470d2bc8d8563e9
paper_authors.avatarUrl: /avatars/04563e6f022f91c58d642c7e14c7d7d0.svg
paper_authors.fullname: Millon Madhur Das
paper_authors.type: user
paper_authors.user: Sleepyhead
paper_authors._id: 66f18738e32a9f85b62c26d7
paper_authors.name: Aman Bansal
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-03T08:31:52.244Z
paper_authors._id: 65c278d447ac0454b612ebdb
paper_authors.avatarUrl: /avatars/63636a78b7ba289260e112603e4ee63c.svg
paper_authors.fullname: Aman Bansal
paper_authors.type: user
paper_authors.user: bansalaman18
paper_authors._id: 66f18738e32a9f85b62c26d8
paper_authors.name: Ankit Raj
paper_authors._id: 66f18738e32a9f85b62c26d9
paper_authors.name: Pawan Goyal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T16:31:51.286Z
paper_authors._id: 66d1436c55c3f7d512051c2e
paper_authors.avatarUrl: /avatars/8ba01c299a244f350fc91d552192f0b5.svg
paper_authors.fullname: Pawan Goyal
paper_authors.type: user
paper_authors.user: pawang-iitkgp
paper_authors._id: 66f18738e32a9f85b62c26da
paper_authors.name: Niloy Ganguly
paper_title: YesBut: A High-Quality Annotated Multimodal Dataset for evaluating
    Satire Comprehension capability of Vision-Language Models
paper_summary: Understanding satire and humor is a challenging task for even current
  Vision-Language models. In this paper, we propose the challenging tasks of
  Satirical Image Detection (detecting whether an image is satirical),
  Understanding (generating the reason behind the image being satirical), and
  Completion (given one half of the image, selecting the other half from 2 given
  options, such that the complete image is satirical) and release a high-quality
  dataset YesBut, consisting of 2547 images, 1084 satirical and 1463
  non-satirical, containing different artistic styles, to evaluate those tasks.
  Each satirical image in the dataset depicts a normal scenario, along with a
  conflicting scenario which is funny or ironic. Despite the success of current
  Vision-Language Models on multimodal tasks such as Visual QA and Image
  Captioning, our benchmarking experiments show that such models perform poorly
  on the proposed tasks on the YesBut Dataset in Zero-Shot Settings w.r.t both
  automated as well as human evaluation. Additionally, we release a dataset of
  119 real, satirical photographs for further research. The dataset and code are
  available at https://github.com/abhi1nandy2/yesbut_dataset.
paper_ai_summary: Current Vision-Language models struggle with satirical image detection, understanding, and completion tasks on a newly released high-quality dataset called YesBut.
paper_ai_keywords: satirical image detection satirical image understanding satirical image completion Vision-Language models multimodal tasks Visual QA Image Captioning Zero-Shot Settings