title: FreeU: Free Lunch in Diffusion U-Net
summary: In this paper, we uncover the untapped potential of diffusion U-Net, which
  serves as a "free lunch" that substantially improves the generation quality on
  the fly. We initially investigate the key contributions of the U-Net
  architecture to the denoising process and identify that its main backbone
  primarily contributes to denoising, whereas its skip connections mainly
  introduce high-frequency features into the decoder module, causing the network
  to overlook the backbone semantics. Capitalizing on this discovery, we propose
  a simple yet effective method-termed "FreeU" - that enhances generation quality
  without additional training or finetuning. Our key insight is to strategically
  re-weight the contributions sourced from the U-Net's skip connections and
  backbone feature maps, to leverage the strengths of both components of the
  U-Net architecture. Promising results on image and video generation tasks
  demonstrate that our FreeU can be readily integrated to existing diffusion
  models, e.g., Stable Diffusion, DreamBooth, ModelScope, Rerender and ReVersion,
  to improve the generation quality with only a few lines of code. All you need
  is to adjust two scaling factors during inference. Project page:
  https://chenyangsi.top/FreeU/.
paper_authors._id: 650bcd0dd4a0852d3c827c73
paper_authors.name: Chenyang Si
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-21T09:32:08.312Z
paper_authors._id: 635f8ed47c05eb9f59963d3a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/635f8ed47c05eb9f59963d3a/uQf4p9N9pSaFy87Wg9v4k.jpeg
paper_authors.fullname: ChenyangSi
paper_authors.type: user
paper_authors.user: ChenyangSi
paper_authors._id: 650bcd0dd4a0852d3c827c74
paper_authors.name: Ziqi Huang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-09-21T07:22:35.208Z
paper_authors._id: 60efe7fa0d920bc7805cada5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60efe7fa0d920bc7805cada5/2LBrJBjSCOP5ilZIpWLHl.png
paper_authors.fullname: Ziqi Huang
paper_authors.type: user
paper_authors.user: Ziqi
paper_authors._id: 650bcd0dd4a0852d3c827c75
paper_authors.name: Yuming Jiang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-21T09:32:28.591Z
paper_authors._id: 629c95b7a5d6f5fe10e6ed45
paper_authors.avatarUrl: /avatars/c9a5f0e7fae3a079d26db266e9ff90a3.svg
paper_authors.fullname: Yuming Jiang
paper_authors.type: user
paper_authors.user: yumingj
paper_authors._id: 650bcd0dd4a0852d3c827c76
paper_authors.name: Ziwei Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-21T09:32:36.944Z
paper_authors._id: 62ab1ac1d48b4d8b048a3473
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
paper_authors.fullname: Ziwei Liu
paper_authors.type: user
paper_authors.user: liuziwei7
paper_title: FreeU: Free Lunch in Diffusion U-Net
paper_summary: In this paper, we uncover the untapped potential of diffusion U-Net, which
  serves as a "free lunch" that substantially improves the generation quality on
  the fly. We initially investigate the key contributions of the U-Net
  architecture to the denoising process and identify that its main backbone
  primarily contributes to denoising, whereas its skip connections mainly
  introduce high-frequency features into the decoder module, causing the network
  to overlook the backbone semantics. Capitalizing on this discovery, we propose
  a simple yet effective method-termed "FreeU" - that enhances generation quality
  without additional training or finetuning. Our key insight is to strategically
  re-weight the contributions sourced from the U-Net's skip connections and
  backbone feature maps, to leverage the strengths of both components of the
  U-Net architecture. Promising results on image and video generation tasks
  demonstrate that our FreeU can be readily integrated to existing diffusion
  models, e.g., Stable Diffusion, DreamBooth, ModelScope, Rerender and ReVersion,
  to improve the generation quality with only a few lines of code. All you need
  is to adjust two scaling factors during inference. Project page:
  https://chenyangsi.top/FreeU/.
paper_ai_keywords: diffusion U-Net denoising process U-Net architecture skip connections backbone feature maps FreeU generation quality Stable Diffusion DreamBooth ModelScope Rerender ReVersion inference scaling factors