FouriScale: A Frequency Perspective on Training-Free High-Resolution
  Image Synthesis
In this study, we delve into the generation of high-resolution images from
pre-trained diffusion models, addressing persistent challenges, such as
repetitive patterns and structural distortions, that emerge when models are
applied beyond their trained resolutions. To address this issue, we introduce
an innovative, training-free approach FouriScale from the perspective of
frequency domain analysis. We replace the original convolutional layers in
pre-trained diffusion models by incorporating a dilation technique along with a
low-pass operation, intending to achieve structural consistency and scale
consistency across resolutions, respectively. Further enhanced by a
padding-then-crop strategy, our method can flexibly handle text-to-image
generation of various aspect ratios. By using the FouriScale as guidance, our
method successfully balances the structural integrity and fidelity of generated
images, achieving an astonishing capacity of arbitrary-size, high-resolution,
and high-quality generation. With its simplicity and compatibility, our method
can provide valuable insights for future explorations into the synthesis of
ultra-high-resolution images. The code will be released at
https://github.com/LeonHLJ/FouriScale.
65fa4422becc987834315f70
Linjiang Huang
claimed_verified
2024-03-22T08:38:16.184Z
65fc7c824d36be78e66ba92d
/avatars/d4a55c820cae533f91724e062427516a.svg
Linjiang Huang
user
LjHuang
65fa4422becc987834315f71
Rongyao Fang
admin_assigned
2024-03-21T13:08:52.385Z
65b8724123d948d884b379b1
/avatars/ce189d1d8d688c17912f9b869035b2d0.svg
Rongyao Fang
user
LucasFang
65fa4422becc987834315f72
Aiping Zhang
claimed_verified
2024-09-26T08:26:24.916Z
66f4cf499af27c74e9ac6201
/avatars/745be730ac988d4137bed0c2f51e2066.svg
zhangaip
user
zhangap
65fa4422becc987834315f73
Guanglu Song
admin_assigned
2024-03-21T13:09:34.955Z
636a2efffbc31ee68fee5c1c
/avatars/21e119f23e1e6fe926567409e74d1a89.svg
Song
user
Guanglu
65fa4422becc987834315f74
Si Liu
65fa4422becc987834315f75
Yu Liu
65fa4422becc987834315f76
Hongsheng Li
admin_assigned
2024-03-21T13:10:00.385Z
65c04e9c27a5fdca81abcbd9
/avatars/12a155683c824fa23da4a9e2bed4f64e.svg
Hongsheng LI
user
hsli-cuhk
FouriScale: A Frequency Perspective on Training-Free High-Resolution
  Image Synthesis
In this study, we delve into the generation of high-resolution images from
pre-trained diffusion models, addressing persistent challenges, such as
repetitive patterns and structural distortions, that emerge when models are
applied beyond their trained resolutions. To address this issue, we introduce
an innovative, training-free approach FouriScale from the perspective of
frequency domain analysis. We replace the original convolutional layers in
pre-trained diffusion models by incorporating a dilation technique along with a
low-pass operation, intending to achieve structural consistency and scale
consistency across resolutions, respectively. Further enhanced by a
padding-then-crop strategy, our method can flexibly handle text-to-image
generation of various aspect ratios. By using the FouriScale as guidance, our
method successfully balances the structural integrity and fidelity of generated
images, achieving an astonishing capacity of arbitrary-size, high-resolution,
and high-quality generation. With its simplicity and compatibility, our method
can provide valuable insights for future explorations into the synthesis of
ultra-high-resolution images. The code will be released at
https://github.com/LeonHLJ/FouriScale.
FouriScale, a frequency domain technique, improves high-resolution image generation from pre-trained diffusion models by enhancing structural and scale consistency across different resolutions.
diffusion models frequency domain convolutional layers dilation low-pass operation padding-then-crop strategy text-to-image generation structural integrity fidelity