title: Automatic Creative Selection with Cross-Modal Matching
summary: Application developers advertise their Apps by creating product pages with
  App images, and bidding on search terms. It is then crucial for App images to
  be highly relevant with the search terms. Solutions to this problem require an
  image-text matching model to predict the quality of the match between the
  chosen image and the search terms. In this work, we present a novel approach to
  matching an App image to search terms based on fine-tuning a pre-trained LXMERT
  model. We show that compared to the CLIP model and a baseline using a
  Transformer model for search terms, and a ResNet model for images, we
  significantly improve the matching accuracy. We evaluate our approach using two
  sets of labels: advertiser associated (image, search term) pairs for a given
  application, and human ratings for the relevance between (image, search term)
  pairs. Our approach achieves 0.96 AUC score for advertiser associated ground
  truth, outperforming the transformer+ResNet baseline and the fine-tuned CLIP
  model by 8% and 14%. For human labeled ground truth, our approach achieves 0.95
  AUC score, outperforming the transformer+ResNet baseline and the fine-tuned
  CLIP model by 16% and 17%.
paper_authors._id: 66332a4b5ddb7702ad419478
paper_authors.name: Alex Kim
paper_authors._id: 66332a4b5ddb7702ad419479
paper_authors.name: Jia Huang
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-05-02T05:53:15.711Z
paper_authors._id: 64b86a14f1f8e6ea584dd4dc
paper_authors.avatarUrl: /avatars/dc365aff617f5dfd75f44d5b013065bc.svg
paper_authors.fullname: Jia Huang
paper_authors.type: user
paper_authors.user: jiahuang-5
paper_authors._id: 66332a4b5ddb7702ad41947a
paper_authors.name: Rob Monarch
paper_authors._id: 66332a4b5ddb7702ad41947b
paper_authors.name: Jerry Kwac
paper_authors._id: 66332a4b5ddb7702ad41947c
paper_authors.name: Anikesh Kamath
paper_authors._id: 66332a4b5ddb7702ad41947d
paper_authors.name: Parmeshwar Khurd
paper_authors._id: 66332a4b5ddb7702ad41947e
paper_authors.name: Kailash Thiyagarajan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-02T07:42:59.039Z
paper_authors._id: 661b2261a12a10bd0d997f77
paper_authors.avatarUrl: /avatars/f924eafa3e2bba1d6e07782aba04f762.svg
paper_authors.fullname: Kailash Thiyagarajan
paper_authors.type: user
paper_authors.user: tkailash
paper_authors._id: 66332a4b5ddb7702ad41947f
paper_authors.name: Goodman Gu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-02T07:43:05.334Z
paper_authors._id: 6472ef5fd9e9598973c6c558
paper_authors.avatarUrl: /avatars/8a184f6c3754813730a7cb7e70a75df7.svg
paper_authors.fullname: Goodman Gu
paper_authors.type: user
paper_authors.user: vivaldi2007
paper_title: Automatic Creative Selection with Cross-Modal Matching
paper_summary: Application developers advertise their Apps by creating product pages with
  App images, and bidding on search terms. It is then crucial for App images to
  be highly relevant with the search terms. Solutions to this problem require an
  image-text matching model to predict the quality of the match between the
  chosen image and the search terms. In this work, we present a novel approach to
  matching an App image to search terms based on fine-tuning a pre-trained LXMERT
  model. We show that compared to the CLIP model and a baseline using a
  Transformer model for search terms, and a ResNet model for images, we
  significantly improve the matching accuracy. We evaluate our approach using two
  sets of labels: advertiser associated (image, search term) pairs for a given
  application, and human ratings for the relevance between (image, search term)
  pairs. Our approach achieves 0.96 AUC score for advertiser associated ground
  truth, outperforming the transformer+ResNet baseline and the fine-tuned CLIP
  model by 8% and 14%. For human labeled ground truth, our approach achieves 0.95
  AUC score, outperforming the transformer+ResNet baseline and the fine-tuned
  CLIP model by 16% and 17%.
paper_ai_keywords: LXMERT image-text matching fine-tuning pre-trained CLIP model Transformer model ResNet model AUC score