Nested Diffusion Processes for Anytime Image Generation
Diffusion models are the current state-of-the-art in image generation,
synthesizing high-quality images by breaking down the generation process into
many fine-grained denoising steps. Despite their good performance, diffusion
models are computationally expensive, requiring many neural function
evaluations (NFEs). In this work, we propose an anytime diffusion-based method
that can generate viable images when stopped at arbitrary times before
completion. Using existing pretrained diffusion models, we show that the
generation scheme can be recomposed as two nested diffusion processes, enabling
fast iterative refinement of a generated image. We use this Nested Diffusion
approach to peek into the generation process and enable flexible scheduling
based on the instantaneous preference of the user. In experiments on ImageNet
and Stable Diffusion-based text-to-image generation, we show, both
qualitatively and quantitatively, that our method's intermediate generation
quality greatly exceeds that of the original diffusion model, while the final
slow generation result remains comparable.
6476b14a02fc644c810b35bd
Noam Elata
6476b14a02fc644c810b35be
Bahjat Kawar
admin_assigned
2023-05-31T08:41:36.837Z
634e99b9d049354d7ee2614a
/avatars/3dc9cfc40e6e18235a9e0a588c4029d0.svg
Bahjat Kawar
user
bahjat-kawar
6476b14a02fc644c810b35bf
Tomer Michaeli
6476b14a02fc644c810b35c0
Michael Elad
Nested Diffusion Processes for Anytime Image Generation
Diffusion models are the current state-of-the-art in image generation,
synthesizing high-quality images by breaking down the generation process into
many fine-grained denoising steps. Despite their good performance, diffusion
models are computationally expensive, requiring many neural function
evaluations (NFEs). In this work, we propose an anytime diffusion-based method
that can generate viable images when stopped at arbitrary times before
completion. Using existing pretrained diffusion models, we show that the
generation scheme can be recomposed as two nested diffusion processes, enabling
fast iterative refinement of a generated image. We use this Nested Diffusion
approach to peek into the generation process and enable flexible scheduling
based on the instantaneous preference of the user. In experiments on ImageNet
and Stable Diffusion-based text-to-image generation, we show, both
qualitatively and quantitatively, that our method's intermediate generation
quality greatly exceeds that of the original diffusion model, while the final
slow generation result remains comparable.
An anytime diffusion method that can generate high-quality images at various stages of completion using nested diffusion processes, outperforming original diffusion models in intermediate quality.
diffusion models ImageNet Stable Diffusion nested diffusion denoising steps neural function evaluations image generation