PixArt-Σ: Weak-to-Strong Training of Diffusion Transformer for 4K
  Text-to-Image Generation
In this paper, we introduce PixArt-\Sigma, a Diffusion Transformer
model~(DiT) capable of directly generating images at 4K resolution.
PixArt-\Sigma represents a significant advancement over its predecessor,
PixArt-\alpha, offering images of markedly higher fidelity and improved
alignment with text prompts. A key feature of PixArt-\Sigma is its training
efficiency. Leveraging the foundational pre-training of PixArt-\alpha, it
evolves from the `weaker' baseline to a `stronger' model via incorporating
higher quality data, a process we term "weak-to-strong training". The
advancements in PixArt-\Sigma are twofold: (1) High-Quality Training Data:
PixArt-\Sigma incorporates superior-quality image data, paired with more
precise and detailed image captions. (2) Efficient Token Compression: we
propose a novel attention module within the DiT framework that compresses both
keys and values, significantly improving efficiency and facilitating
ultra-high-resolution image generation. Thanks to these improvements,
PixArt-\Sigma achieves superior image quality and user prompt adherence
capabilities with significantly smaller model size (0.6B parameters) than
existing text-to-image diffusion models, such as SDXL (2.6B parameters) and SD
Cascade (5.1B parameters). Moreover, PixArt-\Sigma's capability to generate 4K
images supports the creation of high-resolution posters and wallpapers,
efficiently bolstering the production of high-quality visual content in
industries such as film and gaming.
65ea778b570481e38ef5ce40
Junsong Chen
claimed_verified
2024-03-08T08:33:28.553Z
645b5b09bc7518912e1f9733
/avatars/4d35f728b41f93881a9b67c337f4d1df.svg
Chen
user
Lawrence-cj
65ea778b570481e38ef5ce41
Chongjian Ge
claimed_verified
2024-03-11T08:21:03.105Z
620f126891e167b068fa76f8
https://cdn-avatars.huggingface.co/v1/production/uploads/620f126891e167b068fa76f8/NaPyS5lFjgZYJZrWaf0OI.jpeg
ChongjianGE
user
RhettGee
65ea778b570481e38ef5ce42
Enze Xie
admin_assigned
2024-03-08T09:25:50.624Z
616961bdd3f656f79ad18ec1
https://cdn-avatars.huggingface.co/v1/production/uploads/616961bdd3f656f79ad18ec1/NYpxas378lDevRuUDepJJ.jpeg
xieenze
user
xieenze
65ea778b570481e38ef5ce43
Yue Wu
65ea778b570481e38ef5ce44
Lewei Yao
65ea778b570481e38ef5ce45
Xiaozhe Ren
65ea778b570481e38ef5ce46
Zhongdao Wang
65ea778b570481e38ef5ce47
Ping Luo
65ea778b570481e38ef5ce48
Huchuan Lu
65ea778b570481e38ef5ce49
Zhenguo Li
PixArt-Σ: Weak-to-Strong Training of Diffusion Transformer for 4K
  Text-to-Image Generation
In this paper, we introduce PixArt-\Sigma, a Diffusion Transformer
model~(DiT) capable of directly generating images at 4K resolution.
PixArt-\Sigma represents a significant advancement over its predecessor,
PixArt-\alpha, offering images of markedly higher fidelity and improved
alignment with text prompts. A key feature of PixArt-\Sigma is its training
efficiency. Leveraging the foundational pre-training of PixArt-\alpha, it
evolves from the `weaker' baseline to a `stronger' model via incorporating
higher quality data, a process we term "weak-to-strong training". The
advancements in PixArt-\Sigma are twofold: (1) High-Quality Training Data:
PixArt-\Sigma incorporates superior-quality image data, paired with more
precise and detailed image captions. (2) Efficient Token Compression: we
propose a novel attention module within the DiT framework that compresses both
keys and values, significantly improving efficiency and facilitating
ultra-high-resolution image generation. Thanks to these improvements,
PixArt-\Sigma achieves superior image quality and user prompt adherence
capabilities with significantly smaller model size (0.6B parameters) than
existing text-to-image diffusion models, such as SDXL (2.6B parameters) and SD
Cascade (5.1B parameters). Moreover, PixArt-\Sigma's capability to generate 4K
images supports the creation of high-resolution posters and wallpapers,
efficiently bolstering the production of high-quality visual content in
industries such as film and gaming.
PixArt-\Sigma, a Diffusion Transformer model, generates high-quality 4K images more efficiently than existing models using improved data and novel token compression techniques.
Diffusion Transformer DiT high-fidelity images weak-to-strong training high-quality training data efficient token compression ultra-high-resolution image generation attention module text-to-image diffusion models SDXL SD Cascade