title: StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual
    Representation Learners
summary: We investigate the potential of learning visual representations using
  synthetic images generated by text-to-image models. This is a natural question
  in the light of the excellent performance of such models in generating
  high-quality images. We consider specifically the Stable Diffusion, one of the
  leading open source text-to-image models. We show that (1) when the generative
  model is configured with proper classifier-free guidance scale, training
  self-supervised methods on synthetic images can match or beat the real image
  counterpart; (2) by treating the multiple images generated from the same text
  prompt as positives for each other, we develop a multi-positive contrastive
  learning method, which we call StableRep. With solely synthetic images, the
  representations learned by StableRep surpass the performance of representations
  learned by SimCLR and CLIP using the same set of text prompts and corresponding
  real images, on large scale datasets. When we further add language supervision,
  StableRep trained with 20M synthetic images achieves better accuracy than CLIP
  trained with 50M real images.
paper_authors._id: 64797ff450ff7001631a4451
paper_authors.name: Yonglong Tian
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-02T07:30:44.118Z
paper_authors._id: 64793086ac77bb7603842456
paper_authors.avatarUrl: /avatars/5891292138d372a9cc063f86c08b8174.svg
paper_authors.fullname: Yonglong Tian
paper_authors.type: user
paper_authors.user: tyl5566
paper_authors._id: 64797ff450ff7001631a4452
paper_authors.name: Lijie Fan
paper_authors._id: 64797ff450ff7001631a4453
paper_authors.name: Phillip Isola
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-02T07:30:06.506Z
paper_authors._id: 623cb1f3ca87989bbf797eee
paper_authors.avatarUrl: /avatars/13fbd5b123b8676561751f820317600c.svg
paper_authors.fullname: Phillip Isola
paper_authors.type: user
paper_authors.user: phillipi
paper_authors._id: 64797ff450ff7001631a4454
paper_authors.name: Huiwen Chang
paper_authors._id: 64797ff450ff7001631a4455
paper_authors.name: Dilip Krishnan
paper_title: StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual
    Representation Learners
paper_summary: We investigate the potential of learning visual representations using
  synthetic images generated by text-to-image models. This is a natural question
  in the light of the excellent performance of such models in generating
  high-quality images. We consider specifically the Stable Diffusion, one of the
  leading open source text-to-image models. We show that (1) when the generative
  model is configured with proper classifier-free guidance scale, training
  self-supervised methods on synthetic images can match or beat the real image
  counterpart; (2) by treating the multiple images generated from the same text
  prompt as positives for each other, we develop a multi-positive contrastive
  learning method, which we call StableRep. With solely synthetic images, the
  representations learned by StableRep surpass the performance of representations
  learned by SimCLR and CLIP using the same set of text prompts and corresponding
  real images, on large scale datasets. When we further add language supervision,
  StableRep trained with 20M synthetic images achieves better accuracy than CLIP
  trained with 50M real images.
paper_ai_summary: Training self-supervised methods on synthetic images generated by a text-to-image model like Stable Diffusion can yield high-performing visual representations matching or exceeding those trained on real images.
paper_ai_keywords: text-to-image models Stable Diffusion classifier-free guidance scale self-supervised methods multi-positive contrastive learning StableRep SimCLR CLIP large scale datasets language supervision