MegaFusion: Extend Diffusion Models towards Higher-resolution Image
  Generation without Further Tuning
Diffusion models have emerged as frontrunners in text-to-image generation for
their impressive capabilities. Nonetheless, their fixed image resolution during
training often leads to challenges in high-resolution image generation, such as
semantic inaccuracies and object replication. This paper introduces MegaFusion,
a novel approach that extends existing diffusion-based text-to-image generation
models towards efficient higher-resolution generation without additional
fine-tuning or extra adaptation. Specifically, we employ an innovative truncate
and relay strategy to bridge the denoising processes across different
resolutions, allowing for high-resolution image generation in a coarse-to-fine
manner. Moreover, by integrating dilated convolutions and noise re-scheduling,
we further adapt the model's priors for higher resolution. The versatility and
efficacy of MegaFusion make it universally applicable to both latent-space and
pixel-space diffusion models, along with other derivative models. Extensive
experiments confirm that MegaFusion significantly boosts the capability of
existing models to produce images of megapixels and various aspect ratios,
while only requiring about 40% of the original computational cost.
66c54a9336a75deef8a188e9
Haoning Wu
claimed_verified
2024-08-22T19:51:23.864Z
632c7a0d1d303f5f9acf01b8
https://cdn-avatars.huggingface.co/v1/production/uploads/632c7a0d1d303f5f9acf01b8/T010IFuCp6UaOeIyWhbCk.jpeg
Haoning Wu
user
haoningwu
66c54a9336a75deef8a188ea
Shaocheng Shen
66c54a9336a75deef8a188eb
Qiang Hu
66c54a9336a75deef8a188ec
Xiaoyun Zhang
66c54a9336a75deef8a188ed
Ya Zhang
66c54a9336a75deef8a188ee
Yanfeng Wang
MegaFusion: Extend Diffusion Models towards Higher-resolution Image
  Generation without Further Tuning
Diffusion models have emerged as frontrunners in text-to-image generation for
their impressive capabilities. Nonetheless, their fixed image resolution during
training often leads to challenges in high-resolution image generation, such as
semantic inaccuracies and object replication. This paper introduces MegaFusion,
a novel approach that extends existing diffusion-based text-to-image generation
models towards efficient higher-resolution generation without additional
fine-tuning or extra adaptation. Specifically, we employ an innovative truncate
and relay strategy to bridge the denoising processes across different
resolutions, allowing for high-resolution image generation in a coarse-to-fine
manner. Moreover, by integrating dilated convolutions and noise re-scheduling,
we further adapt the model's priors for higher resolution. The versatility and
efficacy of MegaFusion make it universally applicable to both latent-space and
pixel-space diffusion models, along with other derivative models. Extensive
experiments confirm that MegaFusion significantly boosts the capability of
existing models to produce images of megapixels and various aspect ratios,
while only requiring about 40% of the original computational cost.
MegaFusion enhances diffusion-based text-to-image generation for high-resolution images using a truncate and relay strategy with dilated convolutions and noise re-scheduling.
diffusion models text-to-image generation high-resolution image generation truncate and relay strategy dilated convolutions noise re-scheduling latent-space pixel-space computational cost