On the Trajectory Regularity of ODE-based Diffusion Sampling
<|reference_start|>On the Trajectory Regularity of ODE-based Diffusion Sampling: Diffusion-based generative models use stochastic differential equations (SDEs) and their equivalent ordinary differential equations (ODEs) to establish a smooth connection between a complex data distribution and a tractable prior distribution. In this paper, we identify several intriguing trajectory properties in the ODE-based sampling process of diffusion models. We characterize an implicit denoising trajectory and discuss its vital role in forming the coupled sampling trajectory with a strong shape regularity, regardless of the generated content. We also describe a dynamic programming-based scheme to make the time schedule in sampling better fit the underlying trajectory structure. This simple strategy requires minimal modification to any given ODE-based numerical solvers and incurs negligible computational cost, while delivering superior performance in image generation, especially in $5\sim 10$ function evaluations.<|reference_end|>
@article{chen2024on,
    title={On the Trajectory Regularity of ODE-based Diffusion Sampling},
    author={Defang Chen, Zhenyu Zhou, Can Wang, Chunhua Shen, Siwei Lyu},
    journal={arXiv preprint arXiv:2405.11326},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.11326},
    primaryClass={cs.LG cs.CV}
}