FreSca: Unveiling the Scaling Space in Diffusion Models
Diffusion models offer impressive controllability for image tasks, primarily
through noise predictions that encode task-specific information and
classifier-free guidance enabling adjustable scaling. This scaling mechanism
implicitly defines a ``scaling space'' whose potential for fine-grained
semantic manipulation remains underexplored. We investigate this space,
starting with inversion-based editing where the difference between
conditional/unconditional noise predictions carries key semantic information.
Our core contribution stems from a Fourier analysis of noise predictions,
revealing that its low- and high-frequency components evolve differently
throughout diffusion. Based on this insight, we introduce FreSca, a
straightforward method that applies guidance scaling independently to different
frequency bands in the Fourier domain. FreSca demonstrably enhances existing
image editing methods without retraining. Excitingly, its effectiveness extends
to image understanding tasks such as depth estimation, yielding quantitative
gains across multiple datasets.
67efe4bde065b7e7675be166
Chao Huang
claimed_verified
2025-05-21T09:05:18.753Z
67257ee0938e718957c9c100
/avatars/db1f792ee1a5d860ea4e98c11a016a1b.svg
Chao Huang
user
ChaoHuangCS
67efe4bde065b7e7675be167
Susan Liang
67efe4bde065b7e7675be168
Yunlong Tang
67efe4bde065b7e7675be169
Li Ma
67efe4bde065b7e7675be16a
Yapeng Tian
67efe4bde065b7e7675be16b
Chenliang Xu
FreSca: Unveiling the Scaling Space in Diffusion Models
Diffusion models offer impressive controllability for image tasks, primarily
through noise predictions that encode task-specific information and
classifier-free guidance enabling adjustable scaling. This scaling mechanism
implicitly defines a ``scaling space'' whose potential for fine-grained
semantic manipulation remains underexplored. We investigate this space,
starting with inversion-based editing where the difference between
conditional/unconditional noise predictions carries key semantic information.
Our core contribution stems from a Fourier analysis of noise predictions,
revealing that its low- and high-frequency components evolve differently
throughout diffusion. Based on this insight, we introduce FreSca, a
straightforward method that applies guidance scaling independently to different
frequency bands in the Fourier domain. FreSca demonstrably enhances existing
image editing methods without retraining. Excitingly, its effectiveness extends
to image understanding tasks such as depth estimation, yielding quantitative
gains across multiple datasets.
FreSca enhances image editing and understanding tasks by applying independent guidance scaling to different frequency bands in diffusion models.
diffusion models noise predictions scaling space inversion-based editing Fourier analysis frequency bands FreSca image editing image understanding depth estimation quantitative gains