What's the score? Automated Denoising Score Matching for Nonlinear Diffusions
<|reference_start|>What's the score? Automated Denoising Score Matching for Nonlinear Diffusions: Reversing a diffusion process by learning its score forms the heart of diffusion-based generative modeling and for estimating properties of scientific systems. The diffusion processes that are tractable center on linear processes with a Gaussian stationary distribution. This limits the kinds of models that can be built to those that target a Gaussian prior or more generally limits the kinds of problems that can be generically solved to those that have conditionally linear score functions. In this work, we introduce a family of tractable denoising score matching objectives, called local-DSM, built using local increments of the diffusion process. We show how local-DSM melded with Taylor expansions enables automated training and score estimation with nonlinear diffusion processes. To demonstrate these ideas, we use automated-DSM to train generative models using non-Gaussian priors on challenging low dimensional distributions and the CIFAR10 image dataset. Additionally, we use the automated-DSM to learn the scores for nonlinear processes studied in statistical physics.<|reference_end|>
@article{singhal2024what's,
    title={What's the score? Automated Denoising Score Matching for Nonlinear
  Diffusions},
    author={Raghav Singhal, Mark Goldstein, Rajesh Ranganath},
    journal={arXiv preprint arXiv:2407.07998},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.07998},
    primaryClass={cs.LG stat.ML}
}