Through the Looking Glass: Common Sense Consistency Evaluation of Weird
  Images
Measuring how real images look is a complex task in artificial intelligence
research. For example, an image of a boy with a vacuum cleaner in a desert
violates common sense. We introduce a novel method, which we call Through the
Looking Glass (TLG), to assess image common sense consistency using Large
Vision-Language Models (LVLMs) and Transformer-based encoder. By leveraging
LVLMs to extract atomic facts from these images, we obtain a mix of accurate
facts. We proceed by fine-tuning a compact attention-pooling classifier over
encoded atomic facts. Our TLG has achieved a new state-of-the-art performance
on the WHOOPS! and WEIRD datasets while leveraging a compact fine-tuning
component.
682cc6b48b0dfc6de9ed8de0
Elisei Rykov
claimed_verified
2025-05-20T18:25:43.631Z
636b867ecde3707d10999b96
https://cdn-avatars.huggingface.co/v1/production/uploads/1667991162387-noauth.png
Rykov Elisei
user
lmeribal
682cc6b48b0dfc6de9ed8de1
Kseniia Petrushina
claimed_verified
2025-05-20T19:28:55.437Z
6240c68d22da70b02aff46e1
/avatars/43acfacb5679e191859d36476a079251.svg
Kseniia
user
rrevoid
682cc6b48b0dfc6de9ed8de2
Kseniia Titova
claimed_verified
2025-06-24T09:33:10.970Z
6380982dea9dfab22296bb6f
/avatars/664055c2f387b827281b2973a9c0b835.svg
Kseniia
user
sitnich
682cc6b48b0dfc6de9ed8de3
Anton Razzhigaev
682cc6b48b0dfc6de9ed8de4
Alexander Panchenko
682cc6b48b0dfc6de9ed8de5
Vasily Konovalov
claimed_verified
2025-05-20T19:28:57.499Z
622b1f6b9f6139daa8e998ce
/avatars/842719c100a5969be75d04da97333675.svg
Vasily Konovalov
user
Vasily
Through the Looking Glass: Common Sense Consistency Evaluation of Weird
  Images
Measuring how real images look is a complex task in artificial intelligence
research. For example, an image of a boy with a vacuum cleaner in a desert
violates common sense. We introduce a novel method, which we call Through the
Looking Glass (TLG), to assess image common sense consistency using Large
Vision-Language Models (LVLMs) and Transformer-based encoder. By leveraging
LVLMs to extract atomic facts from these images, we obtain a mix of accurate
facts. We proceed by fine-tuning a compact attention-pooling classifier over
encoded atomic facts. Our TLG has achieved a new state-of-the-art performance
on the WHOOPS! and WEIRD datasets while leveraging a compact fine-tuning
component.
A new method, Through the Looking Glass (TLG), uses Large Vision-Language Models and Transformer-based encoders to improve image common sense consistency assessment on the WHOOPS! and WEIRD datasets.
Large Vision-Language Models (LVLMs) Transformer-based encoder attention-pooling classifier WHOOPS! WEIRD datasets