title: Diffusion Models Beat GANs on Image Classification
summary: While many unsupervised learning models focus on one family of tasks, either
  generative or discriminative, we explore the possibility of a unified
  representation learner: a model which uses a single pre-training stage to
  address both families of tasks simultaneously. We identify diffusion models as
  a prime candidate. Diffusion models have risen to prominence as a
  state-of-the-art method for image generation, denoising, inpainting,
  super-resolution, manipulation, etc. Such models involve training a U-Net to
  iteratively predict and remove noise, and the resulting model can synthesize
  high fidelity, diverse, novel images. The U-Net architecture, as a
  convolution-based architecture, generates a diverse set of feature
  representations in the form of intermediate feature maps. We present our
  findings that these embeddings are useful beyond the noise prediction task, as
  they contain discriminative information and can also be leveraged for
  classification. We explore optimal methods for extracting and using these
  embeddings for classification tasks, demonstrating promising results on the
  ImageNet classification task. We find that with careful feature selection and
  pooling, diffusion models outperform comparable generative-discriminative
  methods such as BigBiGAN for classification tasks. We investigate diffusion
  models in the transfer learning regime, examining their performance on several
  fine-grained visual classification datasets. We compare these embeddings to
  those generated by competing architectures and pre-trainings for classification
  tasks.
paper_authors._id: 64b5fc829c750f981bcc8c86
paper_authors.name: Soumik Mukhopadhyay
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-18T15:03:40.753Z
paper_authors._id: 64b67e95d5a982941dd5ead5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64b67e95d5a982941dd5ead5/9Gd8wqqaR0c4W0FbSxb44.jpeg
paper_authors.fullname: Soumik Mukhopadhyay
paper_authors.type: user
paper_authors.user: soumikkanad
paper_authors._id: 64b5fc829c750f981bcc8c87
paper_authors.name: Matthew Gwilliam
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-19T06:28:00.489Z
paper_authors._id: 6250c5008c2cdfaca2a681d9
paper_authors.avatarUrl: /avatars/e4f168396c151c1f8077768430bfc673.svg
paper_authors.fullname: Matt Gwilliam
paper_authors.type: user
paper_authors.user: mgwillia
paper_authors._id: 64b5fc829c750f981bcc8c88
paper_authors.name: Vatsal Agarwal
paper_authors._id: 64b5fc829c750f981bcc8c89
paper_authors.name: Namitha Padmanabhan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-24T07:44:12.908Z
paper_authors._id: 64bde9ff796f20daad7e5560
paper_authors.avatarUrl: /avatars/6941bc63472354066a247598da22fe03.svg
paper_authors.fullname: Namitha Padmanabhan
paper_authors.type: user
paper_authors.user: namithap
paper_authors._id: 64b5fc829c750f981bcc8c8a
paper_authors.name: Archana Swaminathan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-21T19:25:03.984Z
paper_authors._id: 64baae8e94c0e3be4aa2cd0d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64baae8e94c0e3be4aa2cd0d/X64nbllxw3Vn_uy4j0qxY.jpeg
paper_authors.fullname: Archana Swaminathan
paper_authors.type: user
paper_authors.user: archana14
paper_authors._id: 64b5fc829c750f981bcc8c8b
paper_authors.name: Srinidhi Hegde
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-19T06:28:05.330Z
paper_authors._id: 64b6aa9746f9829ba0931a54
paper_authors.avatarUrl: /avatars/55912a88762dee2b87c50aaa8deca206.svg
paper_authors.fullname: Srinidhi Hegde
paper_authors.type: user
paper_authors.user: srihegde
paper_authors._id: 64b5fc829c750f981bcc8c8c
paper_authors.name: Tianyi Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-18T06:43:42.546Z
paper_authors._id: 647f5af5b0e96764589f3b2a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VJ4cDyjp5M3V5WmI5gPIU.jpeg
paper_authors.fullname: Tianyi Zhou
paper_authors.type: user
paper_authors.user: zhoutianyi
paper_authors._id: 64b5fc829c750f981bcc8c8d
paper_authors.name: Abhinav Shrivastava
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-18T15:03:36.998Z
paper_authors._id: 64b683acb20146a0d307222c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/l665j1WeROwhQKgIsszYp.jpeg
paper_authors.fullname: Abhinav Shrivastava
paper_authors.type: user
paper_authors.user: abhi2610
paper_title: Diffusion Models Beat GANs on Image Classification
paper_summary: While many unsupervised learning models focus on one family of tasks, either
  generative or discriminative, we explore the possibility of a unified
  representation learner: a model which uses a single pre-training stage to
  address both families of tasks simultaneously. We identify diffusion models as
  a prime candidate. Diffusion models have risen to prominence as a
  state-of-the-art method for image generation, denoising, inpainting,
  super-resolution, manipulation, etc. Such models involve training a U-Net to
  iteratively predict and remove noise, and the resulting model can synthesize
  high fidelity, diverse, novel images. The U-Net architecture, as a
  convolution-based architecture, generates a diverse set of feature
  representations in the form of intermediate feature maps. We present our
  findings that these embeddings are useful beyond the noise prediction task, as
  they contain discriminative information and can also be leveraged for
  classification. We explore optimal methods for extracting and using these
  embeddings for classification tasks, demonstrating promising results on the
  ImageNet classification task. We find that with careful feature selection and
  pooling, diffusion models outperform comparable generative-discriminative
  methods such as BigBiGAN for classification tasks. We investigate diffusion
  models in the transfer learning regime, examining their performance on several
  fine-grained visual classification datasets. We compare these embeddings to
  those generated by competing architectures and pre-trainings for classification
  tasks.
paper_ai_summary: Diffusion models trained on image generation tasks produce useful feature embeddings that perform well for classification tasks and outperform generative-discriminative models like BigBiGAN.
paper_ai_keywords: diffusion models generative tasks discriminative tasks U-Net feature representations intermediate feature maps classification feature selection pooling transfer learning ImageNet fine-grained visual classification BigBiGAN