Diffusion Models Beat GANs on Image Classification
While many unsupervised learning models focus on one family of tasks, either
generative or discriminative, we explore the possibility of a unified
representation learner: a model which uses a single pre-training stage to
address both families of tasks simultaneously. We identify diffusion models as
a prime candidate. Diffusion models have risen to prominence as a
state-of-the-art method for image generation, denoising, inpainting,
super-resolution, manipulation, etc. Such models involve training a U-Net to
iteratively predict and remove noise, and the resulting model can synthesize
high fidelity, diverse, novel images. The U-Net architecture, as a
convolution-based architecture, generates a diverse set of feature
representations in the form of intermediate feature maps. We present our
findings that these embeddings are useful beyond the noise prediction task, as
they contain discriminative information and can also be leveraged for
classification. We explore optimal methods for extracting and using these
embeddings for classification tasks, demonstrating promising results on the
ImageNet classification task. We find that with careful feature selection and
pooling, diffusion models outperform comparable generative-discriminative
methods such as BigBiGAN for classification tasks. We investigate diffusion
models in the transfer learning regime, examining their performance on several
fine-grained visual classification datasets. We compare these embeddings to
those generated by competing architectures and pre-trainings for classification
tasks.
64b5fc829c750f981bcc8c86
Soumik Mukhopadhyay
claimed_verified
2023-07-18T15:03:40.753Z
64b67e95d5a982941dd5ead5
https://cdn-avatars.huggingface.co/v1/production/uploads/64b67e95d5a982941dd5ead5/9Gd8wqqaR0c4W0FbSxb44.jpeg
Soumik Mukhopadhyay
user
soumikkanad
64b5fc829c750f981bcc8c87
Matthew Gwilliam
claimed_verified
2023-07-19T06:28:00.489Z
6250c5008c2cdfaca2a681d9
/avatars/e4f168396c151c1f8077768430bfc673.svg
Matt Gwilliam
user
mgwillia
64b5fc829c750f981bcc8c88
Vatsal Agarwal
claimed_verified
2025-07-10T07:10:33.580Z
653213eaa0dffcea236ca422
/avatars/69d30ce923edcf99e177669c2f8bf9a5.svg
Vatsal Agarwal
user
vatsalag
64b5fc829c750f981bcc8c89
Namitha Padmanabhan
claimed_verified
2023-07-24T07:44:12.908Z
64bde9ff796f20daad7e5560
/avatars/6941bc63472354066a247598da22fe03.svg
Namitha Padmanabhan
user
namithap
64b5fc829c750f981bcc8c8a
Archana Swaminathan
claimed_verified
2023-07-21T19:25:03.984Z
64baae8e94c0e3be4aa2cd0d
https://cdn-avatars.huggingface.co/v1/production/uploads/64baae8e94c0e3be4aa2cd0d/X64nbllxw3Vn_uy4j0qxY.jpeg
Archana Swaminathan
user
archana14
64b5fc829c750f981bcc8c8b
Srinidhi Hegde
claimed_verified
2023-07-19T06:28:05.330Z
64b6aa9746f9829ba0931a54
/avatars/55912a88762dee2b87c50aaa8deca206.svg
Srinidhi Hegde
user
srihegde
64b5fc829c750f981bcc8c8c
Tianyi Zhou
claimed_verified
2023-07-18T06:43:42.546Z
647f5af5b0e96764589f3b2a
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VJ4cDyjp5M3V5WmI5gPIU.jpeg
Tianyi Zhou
user
zhoutianyi
64b5fc829c750f981bcc8c8d
Abhinav Shrivastava
claimed_verified
2023-07-18T15:03:36.998Z
64b683acb20146a0d307222c
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/l665j1WeROwhQKgIsszYp.jpeg
Abhinav Shrivastava
user
abhi2610
Diffusion Models Beat GANs on Image Classification
While many unsupervised learning models focus on one family of tasks, either
generative or discriminative, we explore the possibility of a unified
representation learner: a model which uses a single pre-training stage to
address both families of tasks simultaneously. We identify diffusion models as
a prime candidate. Diffusion models have risen to prominence as a
state-of-the-art method for image generation, denoising, inpainting,
super-resolution, manipulation, etc. Such models involve training a U-Net to
iteratively predict and remove noise, and the resulting model can synthesize
high fidelity, diverse, novel images. The U-Net architecture, as a
convolution-based architecture, generates a diverse set of feature
representations in the form of intermediate feature maps. We present our
findings that these embeddings are useful beyond the noise prediction task, as
they contain discriminative information and can also be leveraged for
classification. We explore optimal methods for extracting and using these
embeddings for classification tasks, demonstrating promising results on the
ImageNet classification task. We find that with careful feature selection and
pooling, diffusion models outperform comparable generative-discriminative
methods such as BigBiGAN for classification tasks. We investigate diffusion
models in the transfer learning regime, examining their performance on several
fine-grained visual classification datasets. We compare these embeddings to
those generated by competing architectures and pre-trainings for classification
tasks.
Diffusion models trained on image generation tasks produce useful feature embeddings that perform well for classification tasks and outperform generative-discriminative models like BigBiGAN.
diffusion models generative tasks discriminative tasks U-Net feature representations intermediate feature maps classification feature selection pooling transfer learning ImageNet fine-grained visual classification BigBiGAN