title: On the Limitations of Vision-Language Models in Understanding Image
    Transforms
summary: Vision Language Models (VLMs) have demonstrated significant potential in
  various downstream tasks, including Image/Video Generation, Visual Question
  Answering, Multimodal Chatbots, and Video Understanding. However, these models
  often struggle with basic image transformations. This paper investigates the
  image-level understanding of VLMs, specifically CLIP by OpenAI and SigLIP by
  Google. Our findings reveal that these models lack comprehension of multiple
  image-level augmentations. To facilitate this study, we created an augmented
  version of the Flickr8k dataset, pairing each image with a detailed description
  of the applied transformation. We further explore how this deficiency impacts
  downstream tasks, particularly in image editing, and evaluate the performance
  of state-of-the-art Image2Image models on simple transformations.
paper_authors._id: 67d4194fc9f149faff46913c
paper_authors.name: Ahmad Mustafa Anis
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-16T21:14:01.381Z
paper_authors._id: 6246908d8031dcfa9ef6d80b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6246908d8031dcfa9ef6d80b/hVdURjUl1RS2MZf4qOhvI.jpeg
paper_authors.fullname: Ahmad Mustafa Anis
paper_authors.type: user
paper_authors.user: AhmadMustafa
paper_authors._id: 67d4194fc9f149faff46913d
paper_authors.name: Hasnain Ali
paper_authors._id: 67d4194fc9f149faff46913e
paper_authors.name: Saquib Sarfraz
paper_title: On the Limitations of Vision-Language Models in Understanding Image
    Transforms
paper_summary: Vision Language Models (VLMs) have demonstrated significant potential in
  various downstream tasks, including Image/Video Generation, Visual Question
  Answering, Multimodal Chatbots, and Video Understanding. However, these models
  often struggle with basic image transformations. This paper investigates the
  image-level understanding of VLMs, specifically CLIP by OpenAI and SigLIP by
  Google. Our findings reveal that these models lack comprehension of multiple
  image-level augmentations. To facilitate this study, we created an augmented
  version of the Flickr8k dataset, pairing each image with a detailed description
  of the applied transformation. We further explore how this deficiency impacts
  downstream tasks, particularly in image editing, and evaluate the performance
  of state-of-the-art Image2Image models on simple transformations.
paper_ai_summary: Studies on Vision Language Models, such as CLIP and SigLIP, show poor understanding of image augmentations, impacting tasks like image editing, as evidenced by evaluations on modified Image2Image models.
paper_ai_keywords: Vision Language Models CLIP SigLIP image-level augmentations downstream tasks image editing Image2Image models