title: A Noise is Worth Diffusion Guidance
summary: Diffusion models excel in generating high-quality images. However, current
  diffusion models struggle to produce reliable images without guidance methods,
  such as classifier-free guidance (CFG). Are guidance methods truly necessary?
  Observing that noise obtained via diffusion inversion can reconstruct
  high-quality images without guidance, we focus on the initial noise of the
  denoising pipeline. By mapping Gaussian noise to `guidance-free noise', we
  uncover that small low-magnitude low-frequency components significantly enhance
  the denoising process, removing the need for guidance and thus improving both
  inference throughput and memory. Expanding on this, we propose \ours, a novel
  method that replaces guidance methods with a single refinement of the initial
  noise. This refined noise enables high-quality image generation without
  guidance, within the same diffusion pipeline. Our noise-refining model
  leverages efficient noise-space learning, achieving rapid convergence and
  strong performance with just 50K text-image pairs. We validate its
  effectiveness across diverse metrics and analyze how refined noise can
  eliminate the need for guidance. See our project page:
  https://cvlab-kaist.github.io/NoiseRefine/.
paper_authors._id: 6752a5285281c3cae4a81f18
paper_authors.name: Donghoon Ahn
paper_authors._id: 6752a5285281c3cae4a81f19
paper_authors.name: Jiwon Kang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-06T17:17:33.670Z
paper_authors._id: 64bb4f6265b648b2df793ded
paper_authors.avatarUrl: /avatars/fb384a47a4a2fa0c2c6319d0a8ade0b8.svg
paper_authors.fullname: Jiwon Kang
paper_authors.type: user
paper_authors.user: ji1kang
paper_authors._id: 6752a5285281c3cae4a81f1a
paper_authors.name: Sanghyun Lee
paper_authors._id: 6752a5285281c3cae4a81f1b
paper_authors.name: Jaewon Min
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-06T10:37:52.730Z
paper_authors._id: 66012e9c9e1cf5eb41ee0c4c
paper_authors.avatarUrl: /avatars/b07240cb86315b9e33d14677e02e4024.svg
paper_authors.fullname: Jaewon Min
paper_authors.type: user
paper_authors.user: Min-Jaewon
paper_authors._id: 6752a5285281c3cae4a81f1c
paper_authors.name: Minjae Kim
paper_authors._id: 6752a5285281c3cae4a81f1d
paper_authors.name: Wooseok Jang
paper_authors._id: 6752a5285281c3cae4a81f1e
paper_authors.name: Hyoungwon Cho
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-06T17:16:07.961Z
paper_authors._id: 6601282b569b30694e67b886
paper_authors.avatarUrl: /avatars/bd6448ed822362b1470e91b749b920be.svg
paper_authors.fullname: Hyoungwon Cho
paper_authors.type: user
paper_authors.user: hyoungwoncho
paper_authors._id: 6752a5285281c3cae4a81f1f
paper_authors.name: Sayak Paul
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-06T17:15:43.220Z
paper_authors._id: 5f7fbd813e94f16a85448745
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1649681653581-5f7fbd813e94f16a85448745.jpeg
paper_authors.fullname: Sayak Paul
paper_authors.type: user
paper_authors.user: sayakpaul
paper_authors._id: 6752a5285281c3cae4a81f20
paper_authors.name: SeonHwa Kim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-06T17:15:36.626Z
paper_authors._id: 65ebb42e6b0697f087f1c443
paper_authors.avatarUrl: /avatars/b19c15dbae55c08da9f472bde4073ada.svg
paper_authors.fullname: Seonhwa Kim
paper_authors.type: user
paper_authors.user: seonhwak
paper_authors._id: 6752a5285281c3cae4a81f21
paper_authors.name: Eunju Cha
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-06T17:15:30.424Z
paper_authors._id: 64d4d7e5badf1110f768cbe1
paper_authors.avatarUrl: /avatars/3c67f3cbcb610dc906db59b6b73c16b8.svg
paper_authors.fullname: Eunju Cha
paper_authors.type: user
paper_authors.user: ejcha
paper_authors._id: 6752a5285281c3cae4a81f22
paper_authors.name: Kyong Hwan Jin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-06T17:15:13.536Z
paper_authors._id: 6721a856329fc46a399cc988
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/nVNdvInvUw8_qvU8MY7XW.png
paper_authors.fullname: Kyong Hwan Jin
paper_authors.type: user
paper_authors.user: panakino
paper_authors._id: 6752a5285281c3cae4a81f23
paper_authors.name: Seungryong Kim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-06T17:15:06.238Z
paper_authors._id: 65cf717450818a335a1d3021
paper_authors.avatarUrl: /avatars/382a0e0f40f661cda1b2531e3e6ea2ee.svg
paper_authors.fullname: Seungryong Kim
paper_authors.type: user
paper_authors.user: seungryong
paper_title: A Noise is Worth Diffusion Guidance
paper_summary: Diffusion models excel in generating high-quality images. However, current
  diffusion models struggle to produce reliable images without guidance methods,
  such as classifier-free guidance (CFG). Are guidance methods truly necessary?
  Observing that noise obtained via diffusion inversion can reconstruct
  high-quality images without guidance, we focus on the initial noise of the
  denoising pipeline. By mapping Gaussian noise to `guidance-free noise', we
  uncover that small low-magnitude low-frequency components significantly enhance
  the denoising process, removing the need for guidance and thus improving both
  inference throughput and memory. Expanding on this, we propose \ours, a novel
  method that replaces guidance methods with a single refinement of the initial
  noise. This refined noise enables high-quality image generation without
  guidance, within the same diffusion pipeline. Our noise-refining model
  leverages efficient noise-space learning, achieving rapid convergence and
  strong performance with just 50K text-image pairs. We validate its
  effectiveness across diverse metrics and analyze how refined noise can
  eliminate the need for guidance. See our project page:
  https://cvlab-kaist.github.io/NoiseRefine/.
paper_ai_summary: A noise-refining method is introduced to eliminate the need for guidance in diffusion models, enhancing image generation speed and memory efficiency.
paper_ai_keywords: diffusion models classifier-free guidance diffusion inversion denoising pipeline Gaussian noise efficient noise-space learning