title: KAFA: Rethinking Image Ad Understanding with Knowledge-Augmented Feature
    Adaptation of Vision-Language Models
summary: Image ad understanding is a crucial task with wide real-world applications.
  Although highly challenging with the involvement of diverse atypical scenes,
  real-world entities, and reasoning over scene-texts, how to interpret image ads
  is relatively under-explored, especially in the era of foundational
  vision-language models (VLMs) featuring impressive generalizability and
  adaptability. In this paper, we perform the first empirical study of image ad
  understanding through the lens of pre-trained VLMs. We benchmark and reveal
  practical challenges in adapting these VLMs to image ad understanding. We
  propose a simple feature adaptation strategy to effectively fuse multimodal
  information for image ads and further empower it with knowledge of real-world
  entities. We hope our study draws more attention to image ad understanding
  which is broadly relevant to the advertising industry.
paper_authors._id: 6476bf179db7d9fe2ad83f55
paper_authors.name: Zhiwei Jia
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-31T18:31:40.907Z
paper_authors._id: 62d50bf25654195611dba128
paper_authors.avatarUrl: /avatars/8f8345ae934406cb3a6ab7d391b4b5e1.svg
paper_authors.fullname: Zhiwei Jia
paper_authors.type: user
paper_authors.user: SeanJia
paper_authors._id: 6476bf179db7d9fe2ad83f56
paper_authors.name: Pradyumna Narayana
paper_authors._id: 6476bf179db7d9fe2ad83f57
paper_authors.name: Arjun R. Akula
paper_authors._id: 6476bf179db7d9fe2ad83f58
paper_authors.name: Garima Pruthi
paper_authors._id: 6476bf179db7d9fe2ad83f59
paper_authors.name: Hao Su
paper_authors._id: 6476bf179db7d9fe2ad83f5a
paper_authors.name: Sugato Basu
paper_authors._id: 6476bf179db7d9fe2ad83f5b
paper_authors.name: Varun Jampani
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-02T12:35:33.874Z
paper_authors._id: 630e7cfe3fc17ffc50f6e835
paper_authors.avatarUrl: /avatars/0b742ff094a09f9374fafcd97ab9e002.svg
paper_authors.fullname: Varun Jampani
paper_authors.type: user
paper_authors.user: varunjampani
paper_title: KAFA: Rethinking Image Ad Understanding with Knowledge-Augmented Feature
    Adaptation of Vision-Language Models
paper_summary: Image ad understanding is a crucial task with wide real-world applications.
  Although highly challenging with the involvement of diverse atypical scenes,
  real-world entities, and reasoning over scene-texts, how to interpret image ads
  is relatively under-explored, especially in the era of foundational
  vision-language models (VLMs) featuring impressive generalizability and
  adaptability. In this paper, we perform the first empirical study of image ad
  understanding through the lens of pre-trained VLMs. We benchmark and reveal
  practical challenges in adapting these VLMs to image ad understanding. We
  propose a simple feature adaptation strategy to effectively fuse multimodal
  information for image ads and further empower it with knowledge of real-world
  entities. We hope our study draws more attention to image ad understanding
  which is broadly relevant to the advertising industry.
paper_ai_summary: An empirical study of adapting pre-trained vision-language models to understand image ads reveals practical challenges and proposes a feature adaptation strategy to fuse multimodal information and leverage real-world knowledge.
paper_ai_keywords: vision-language models image ad understanding multimodal information