Ultra-Resolution Adaptation with Ease
Text-to-image diffusion models have achieved remarkable progress in recent
years. However, training models for high-resolution image generation remains
challenging, particularly when training data and computational resources are
limited. In this paper, we explore this practical problem from two key
perspectives: data and parameter efficiency, and propose a set of key
guidelines for ultra-resolution adaptation termed URAE. For data
efficiency, we theoretically and empirically demonstrate that synthetic data
generated by some teacher models can significantly promote training
convergence. For parameter efficiency, we find that tuning minor components of
the weight matrices outperforms widely-used low-rank adapters when synthetic
data are unavailable, offering substantial performance gains while maintaining
efficiency. Additionally, for models leveraging guidance distillation, such as
FLUX, we show that disabling classifier-free guidance, i.e., setting
the guidance scale to 1 during adaptation, is crucial for satisfactory
performance. Extensive experiments validate that URAE achieves comparable
2K-generation performance to state-of-the-art closed-source models like FLUX1.1
[Pro] Ultra with only 3K samples and 2K iterations, while setting new
benchmarks for 4K-resolution generation. Codes are available
https://github.com/Huage001/URAE{here}.
67dce4c10784200359ab2494
Ruonan Yu
67dce4c10784200359ab2495
Songhua Liu
67dce4c10784200359ab2496
Zhenxiong Tan
67dce4c10784200359ab2497
Xinchao Wang
Ultra-Resolution Adaptation with Ease
Text-to-image diffusion models have achieved remarkable progress in recent
years. However, training models for high-resolution image generation remains
challenging, particularly when training data and computational resources are
limited. In this paper, we explore this practical problem from two key
perspectives: data and parameter efficiency, and propose a set of key
guidelines for ultra-resolution adaptation termed URAE. For data
efficiency, we theoretically and empirically demonstrate that synthetic data
generated by some teacher models can significantly promote training
convergence. For parameter efficiency, we find that tuning minor components of
the weight matrices outperforms widely-used low-rank adapters when synthetic
data are unavailable, offering substantial performance gains while maintaining
efficiency. Additionally, for models leveraging guidance distillation, such as
FLUX, we show that disabling classifier-free guidance, i.e., setting
the guidance scale to 1 during adaptation, is crucial for satisfactory
performance. Extensive experiments validate that URAE achieves comparable
2K-generation performance to state-of-the-art closed-source models like FLUX1.1
[Pro] Ultra with only 3K samples and 2K iterations, while setting new
benchmarks for 4K-resolution generation. Codes are available
https://github.com/Huage001/URAE{here}.
URAE guidelines improve data and parameter efficiency in ultra-resolution text-to-image diffusion models, achieving high-resolution generation performance with limited data and iterations.
synthetic data weight matrices low-rank adapters classifier-free guidance text-to-image diffusion models ultra-resolution adaptation