title: CatLIP: CLIP-level Visual Recognition Accuracy with 2.7x Faster
    Pre-training on Web-scale Image-Text Data
summary: Contrastive learning has emerged as a transformative method for learning
  effective visual representations through the alignment of image and text
  embeddings. However, pairwise similarity computation in contrastive loss
  between image and text pairs poses computational challenges. This paper
  presents a novel weakly supervised pre-training of vision models on web-scale
  image-text data. The proposed method reframes pre-training on image-text data
  as a classification task. Consequently, it eliminates the need for pairwise
  similarity computations in contrastive loss, achieving a remarkable 2.7times
  acceleration in training speed compared to contrastive learning on web-scale
  data. Through extensive experiments spanning diverse vision tasks, including
  detection and segmentation, we demonstrate that the proposed method maintains
  high representation quality. Our source code along with pre-trained model
  weights and training recipes is available at
  https://github.com/apple/corenet.
paper_authors._id: 662a9a5b565dd1ec4801530a
paper_authors.name: Sachin Mehta
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T12:06:04.810Z
paper_authors._id: 64f792652003abb61bf5c1b8
paper_authors.avatarUrl: /avatars/9721810de3519ece50b059c154baa9ca.svg
paper_authors.fullname: Sachin Mehta
paper_authors.type: user
paper_authors.user: sacmehta
paper_authors._id: 662a9a5b565dd1ec4801530b
paper_authors.name: Maxwell Horton
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T12:06:12.519Z
paper_authors._id: 647a5afc732a0cd866eb512f
paper_authors.avatarUrl: /avatars/fbd0ba2a7ba70414ac903e583ce62933.svg
paper_authors.fullname: Maxwell Horton
paper_authors.type: user
paper_authors.user: mchorton
paper_authors._id: 662a9a5b565dd1ec4801530c
paper_authors.name: Fartash Faghri
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T12:06:18.620Z
paper_authors._id: 65f8945a3ba58880f9794a11
paper_authors.avatarUrl: /avatars/fde0ea1c20f89b3a83dbefb24e728d61.svg
paper_authors.fullname: Fartash Faghri
paper_authors.type: user
paper_authors.user: fartashf
paper_authors._id: 662a9a5b565dd1ec4801530d
paper_authors.name: Mohammad Hossein Sekhavat
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T12:06:25.187Z
paper_authors._id: 65ca91986938a6a81fd80d02
paper_authors.avatarUrl: /avatars/efac7945ac1df4e8949e77465ef0dcbd.svg
paper_authors.fullname: Mohammad Hossein Sekhavat
paper_authors.type: user
paper_authors.user: msekhavat
paper_authors._id: 662a9a5b565dd1ec4801530e
paper_authors.name: Mahyar Najibi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T12:06:47.100Z
paper_authors._id: 661d992d69a65c0fb0717013
paper_authors.avatarUrl: /avatars/9d1f20baff742de90a50232db54d2407.svg
paper_authors.fullname: Mahyar Najibi
paper_authors.type: user
paper_authors.user: mahyar-najibi
paper_authors._id: 662a9a5b565dd1ec4801530f
paper_authors.name: Mehrdad Farajtabar
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T12:06:55.648Z
paper_authors._id: 62d188a6d50433c35eb14020
paper_authors.avatarUrl: /avatars/fc3d149a27bc8396010a2a02148e8ca0.svg
paper_authors.fullname: Mehrdad Farajtabar
paper_authors.type: user
paper_authors.user: mfarajtabar
paper_authors._id: 662a9a5b565dd1ec48015310
paper_authors.name: Oncel Tuzel
paper_authors._id: 662a9a5b565dd1ec48015311
paper_authors.name: Mohammad Rastegari
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T12:07:05.928Z
paper_authors._id: 65d4c7734c2163d05eab4110
paper_authors.avatarUrl: /avatars/b116a9aca7b0a2a4836a29cb4f069d42.svg
paper_authors.fullname: Mohammad Rastegari
paper_authors.type: user
paper_authors.user: mrastegari
paper_title: CatLIP: CLIP-level Visual Recognition Accuracy with 2.7x Faster
    Pre-training on Web-scale Image-Text Data
paper_summary: Contrastive learning has emerged as a transformative method for learning
  effective visual representations through the alignment of image and text
  embeddings. However, pairwise similarity computation in contrastive loss
  between image and text pairs poses computational challenges. This paper
  presents a novel weakly supervised pre-training of vision models on web-scale
  image-text data. The proposed method reframes pre-training on image-text data
  as a classification task. Consequently, it eliminates the need for pairwise
  similarity computations in contrastive loss, achieving a remarkable 2.7times
  acceleration in training speed compared to contrastive learning on web-scale
  data. Through extensive experiments spanning diverse vision tasks, including
  detection and segmentation, we demonstrate that the proposed method maintains
  high representation quality. Our source code along with pre-trained model
  weights and training recipes is available at
  https://github.com/apple/corenet.
paper_ai_summary: A novel weakly supervised pre-training method for vision models on web-scale data reframes the task as classification to avoid pairwise similarity computations, significantly speeding up training while maintaining high representation quality.
paper_ai_keywords: contrastive learning image-text embeddings contrastive loss weakly supervised pre-training classification task