Classification Diffusion Models: Revitalizing Density Ratio Estimation
<|reference_start|>Classification Diffusion Models: Revitalizing Density Ratio Estimation: A prominent family of methods for learning data distributions relies on density ratio estimation (DRE), where a model is trained to $\textit{classify}$ between data samples and samples from some reference distribution. DRE-based models can directly output the likelihood for any given input, a highly desired property that is lacking in most generative techniques. Nevertheless, to date, DRE methods have struggled to accurately capture the distributions of complex high-dimensional data like images, which led to reduced research attention over the years. In this work we present $\textit{classification diffusion models}$ (CDMs), a DRE-based generative method that adopts the formalism of denoising diffusion models (DDMs) while making use of a classifier that predicts the level of noise added to a clean signal. Our method is based on an analytical connection that we derive between an MSE-optimal denoiser for white Gaussian noise and a cross-entropy-optimal classifier for predicting the noise level. To the best of our knowledge, our method is the first DRE-based technique that can successfully generate images. Furthermore, it can output the likelihood of any input in a single forward pass, achieving state-of-the-art negative log likelihood (NLL) among methods with this property. Code is available on the project's webpage in https://shaharYadin.github.io/CDM/ .<|reference_end|>
@article{yadin2024classification,
    title={Classification Diffusion Models: Revitalizing Density Ratio Estimation},
    author={Shahar Yadin, Noam Elata, Tomer Michaeli},
    journal={arXiv preprint arXiv:2402.10095},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.10095},
    primaryClass={cs.LG}
}