Fast Samplers for Inverse Problems in Iterative Refinement Models
<|reference_start|>Fast Samplers for Inverse Problems in Iterative Refinement Models: Constructing fast samplers for unconditional diffusion and flow-matching models has received much attention recently; however, existing methods for solving inverse problems, such as super-resolution, inpainting, or deblurring, still require hundreds to thousands of iterative steps to obtain high-quality results. We propose a plug-and-play framework for constructing efficient samplers for inverse problems, requiring only pre-trained diffusion or flow-matching models. We present Conditional Conjugate Integrators, which leverage the specific form of the inverse problem to project the respective conditional diffusion/flow dynamics into a more amenable space for sampling. Our method complements popular posterior approximation methods for solving inverse problems using diffusion/flow models. We evaluate the proposed method's performance on various linear image restoration tasks across multiple datasets, employing diffusion and flow-matching models. Notably, on challenging inverse problems like 4$\times$ super-resolution on the ImageNet dataset, our method can generate high-quality samples in as few as 5 conditional sampling steps and outperforms competing baselines requiring 20-1000 steps. Our code and models will be publicly available at https://github.com/mandt-lab/CI2RM.<|reference_end|>
@article{pandey2024fast,
    title={Fast Samplers for Inverse Problems in Iterative Refinement Models},
    author={Kushagra Pandey, Ruihan Yang, Stephan Mandt},
    journal={arXiv preprint arXiv:2405.17673},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.17673},
    primaryClass={cs.CV cs.LG stat.ML}
}