title: Through the Looking Glass: Common Sense Consistency Evaluation of Weird
    Images
summary: Measuring how real images look is a complex task in artificial intelligence
  research. For example, an image of a boy with a vacuum cleaner in a desert
  violates common sense. We introduce a novel method, which we call Through the
  Looking Glass (TLG), to assess image common sense consistency using Large
  Vision-Language Models (LVLMs) and Transformer-based encoder. By leveraging
  LVLMs to extract atomic facts from these images, we obtain a mix of accurate
  facts. We proceed by fine-tuning a compact attention-pooling classifier over
  encoded atomic facts. Our TLG has achieved a new state-of-the-art performance
  on the WHOOPS! and WEIRD datasets while leveraging a compact fine-tuning
  component.
paper_authors._id: 682cc6b48b0dfc6de9ed8de0
paper_authors.name: Elisei Rykov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-20T18:25:43.631Z
paper_authors._id: 636b867ecde3707d10999b96
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1667991162387-noauth.png
paper_authors.fullname: Rykov Elisei
paper_authors.type: user
paper_authors.user: lmeribal
paper_authors._id: 682cc6b48b0dfc6de9ed8de1
paper_authors.name: Kseniia Petrushina
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-20T19:28:55.437Z
paper_authors._id: 6240c68d22da70b02aff46e1
paper_authors.avatarUrl: /avatars/43acfacb5679e191859d36476a079251.svg
paper_authors.fullname: Kseniia
paper_authors.type: user
paper_authors.user: rrevoid
paper_authors._id: 682cc6b48b0dfc6de9ed8de2
paper_authors.name: Kseniia Titova
paper_authors._id: 682cc6b48b0dfc6de9ed8de3
paper_authors.name: Anton Razzhigaev
paper_authors._id: 682cc6b48b0dfc6de9ed8de4
paper_authors.name: Alexander Panchenko
paper_authors._id: 682cc6b48b0dfc6de9ed8de5
paper_authors.name: Vasily Konovalov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-20T19:28:57.499Z
paper_authors._id: 622b1f6b9f6139daa8e998ce
paper_authors.avatarUrl: /avatars/842719c100a5969be75d04da97333675.svg
paper_authors.fullname: Vasily Konovalov
paper_authors.type: user
paper_authors.user: Vasily
paper_title: Through the Looking Glass: Common Sense Consistency Evaluation of Weird
    Images
paper_summary: Measuring how real images look is a complex task in artificial intelligence
  research. For example, an image of a boy with a vacuum cleaner in a desert
  violates common sense. We introduce a novel method, which we call Through the
  Looking Glass (TLG), to assess image common sense consistency using Large
  Vision-Language Models (LVLMs) and Transformer-based encoder. By leveraging
  LVLMs to extract atomic facts from these images, we obtain a mix of accurate
  facts. We proceed by fine-tuning a compact attention-pooling classifier over
  encoded atomic facts. Our TLG has achieved a new state-of-the-art performance
  on the WHOOPS! and WEIRD datasets while leveraging a compact fine-tuning
  component.
paper_ai_summary: A new method, Through the Looking Glass (TLG), uses Large Vision-Language Models and Transformer-based encoders to improve image common sense consistency assessment on the WHOOPS! and WEIRD datasets.
paper_ai_keywords: Large Vision-Language Models (LVLMs) Transformer-based encoder attention-pooling classifier WHOOPS! WEIRD datasets