title: PixArt-Σ: Weak-to-Strong Training of Diffusion Transformer for 4K
    Text-to-Image Generation
summary: In this paper, we introduce PixArt-\Sigma, a Diffusion Transformer
  model~(DiT) capable of directly generating images at 4K resolution.
  PixArt-\Sigma represents a significant advancement over its predecessor,
  PixArt-\alpha, offering images of markedly higher fidelity and improved
  alignment with text prompts. A key feature of PixArt-\Sigma is its training
  efficiency. Leveraging the foundational pre-training of PixArt-\alpha, it
  evolves from the `weaker' baseline to a `stronger' model via incorporating
  higher quality data, a process we term "weak-to-strong training". The
  advancements in PixArt-\Sigma are twofold: (1) High-Quality Training Data:
  PixArt-\Sigma incorporates superior-quality image data, paired with more
  precise and detailed image captions. (2) Efficient Token Compression: we
  propose a novel attention module within the DiT framework that compresses both
  keys and values, significantly improving efficiency and facilitating
  ultra-high-resolution image generation. Thanks to these improvements,
  PixArt-\Sigma achieves superior image quality and user prompt adherence
  capabilities with significantly smaller model size (0.6B parameters) than
  existing text-to-image diffusion models, such as SDXL (2.6B parameters) and SD
  Cascade (5.1B parameters). Moreover, PixArt-\Sigma's capability to generate 4K
  images supports the creation of high-resolution posters and wallpapers,
  efficiently bolstering the production of high-quality visual content in
  industries such as film and gaming.
paper_authors._id: 65ea778b570481e38ef5ce40
paper_authors.name: Junsong Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-08T08:33:28.553Z
paper_authors._id: 645b5b09bc7518912e1f9733
paper_authors.avatarUrl: /avatars/4d35f728b41f93881a9b67c337f4d1df.svg
paper_authors.fullname: Chen
paper_authors.type: user
paper_authors.user: Lawrence-cj
paper_authors._id: 65ea778b570481e38ef5ce41
paper_authors.name: Chongjian Ge
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-11T08:21:03.105Z
paper_authors._id: 620f126891e167b068fa76f8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/620f126891e167b068fa76f8/NaPyS5lFjgZYJZrWaf0OI.jpeg
paper_authors.fullname: ChongjianGE
paper_authors.type: user
paper_authors.user: RhettGee
paper_authors._id: 65ea778b570481e38ef5ce42
paper_authors.name: Enze Xie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-08T09:25:50.624Z
paper_authors._id: 616961bdd3f656f79ad18ec1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/616961bdd3f656f79ad18ec1/NYpxas378lDevRuUDepJJ.jpeg
paper_authors.fullname: xieenze
paper_authors.type: user
paper_authors.user: xieenze
paper_authors._id: 65ea778b570481e38ef5ce43
paper_authors.name: Yue Wu
paper_authors._id: 65ea778b570481e38ef5ce44
paper_authors.name: Lewei Yao
paper_authors._id: 65ea778b570481e38ef5ce45
paper_authors.name: Xiaozhe Ren
paper_authors._id: 65ea778b570481e38ef5ce46
paper_authors.name: Zhongdao Wang
paper_authors._id: 65ea778b570481e38ef5ce47
paper_authors.name: Ping Luo
paper_authors._id: 65ea778b570481e38ef5ce48
paper_authors.name: Huchuan Lu
paper_authors._id: 65ea778b570481e38ef5ce49
paper_authors.name: Zhenguo Li
paper_title: PixArt-Σ: Weak-to-Strong Training of Diffusion Transformer for 4K
    Text-to-Image Generation
paper_summary: In this paper, we introduce PixArt-\Sigma, a Diffusion Transformer
  model~(DiT) capable of directly generating images at 4K resolution.
  PixArt-\Sigma represents a significant advancement over its predecessor,
  PixArt-\alpha, offering images of markedly higher fidelity and improved
  alignment with text prompts. A key feature of PixArt-\Sigma is its training
  efficiency. Leveraging the foundational pre-training of PixArt-\alpha, it
  evolves from the `weaker' baseline to a `stronger' model via incorporating
  higher quality data, a process we term "weak-to-strong training". The
  advancements in PixArt-\Sigma are twofold: (1) High-Quality Training Data:
  PixArt-\Sigma incorporates superior-quality image data, paired with more
  precise and detailed image captions. (2) Efficient Token Compression: we
  propose a novel attention module within the DiT framework that compresses both
  keys and values, significantly improving efficiency and facilitating
  ultra-high-resolution image generation. Thanks to these improvements,
  PixArt-\Sigma achieves superior image quality and user prompt adherence
  capabilities with significantly smaller model size (0.6B parameters) than
  existing text-to-image diffusion models, such as SDXL (2.6B parameters) and SD
  Cascade (5.1B parameters). Moreover, PixArt-\Sigma's capability to generate 4K
  images supports the creation of high-resolution posters and wallpapers,
  efficiently bolstering the production of high-quality visual content in
  industries such as film and gaming.
paper_ai_summary: PixArt-\Sigma, a Diffusion Transformer model, generates high-quality 4K images more efficiently than existing models using improved data and novel token compression techniques.
paper_ai_keywords: Diffusion Transformer DiT high-fidelity images weak-to-strong training high-quality training data efficient token compression ultra-high-resolution image generation attention module text-to-image diffusion models SDXL SD Cascade