title: PIXART-δ: Fast and Controllable Image Generation with Latent
    Consistency Models
summary: This technical report introduces PIXART-{\delta}, a text-to-image synthesis
  framework that integrates the Latent Consistency Model (LCM) and ControlNet
  into the advanced PIXART-{\alpha} model. PIXART-{\alpha} is recognized for its
  ability to generate high-quality images of 1024px resolution through a
  remarkably efficient training process. The integration of LCM in
  PIXART-{\delta} significantly accelerates the inference speed, enabling the
  production of high-quality images in just 2-4 steps. Notably, PIXART-{\delta}
  achieves a breakthrough 0.5 seconds for generating 1024x1024 pixel images,
  marking a 7x improvement over the PIXART-{\alpha}. Additionally,
  PIXART-{\delta} is designed to be efficiently trainable on 32GB V100 GPUs
  within a single day. With its 8-bit inference capability (von Platen et al.,
  2023), PIXART-{\delta} can synthesize 1024px images within 8GB GPU memory
  constraints, greatly enhancing its usability and accessibility. Furthermore,
  incorporating a ControlNet-like module enables fine-grained control over
  text-to-image diffusion models. We introduce a novel ControlNet-Transformer
  architecture, specifically tailored for Transformers, achieving explicit
  controllability alongside high-quality image generation. As a state-of-the-art,
  open-source image generation model, PIXART-{\delta} offers a promising
  alternative to the Stable Diffusion family of models, contributing
  significantly to text-to-image synthesis.
paper_authors._id: 659f529fd56cbade8db370aa
paper_authors.name: Junsong Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-11T10:37:30.196Z
paper_authors._id: 645b5b09bc7518912e1f9733
paper_authors.avatarUrl: /avatars/4d35f728b41f93881a9b67c337f4d1df.svg
paper_authors.fullname: Chen
paper_authors.type: user
paper_authors.user: Lawrence-cj
paper_authors._id: 659f529fd56cbade8db370ab
paper_authors.name: Yue Wu
paper_authors._id: 659f529fd56cbade8db370ac
paper_authors.name: Simian Luo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-11T10:33:40.265Z
paper_authors._id: 652a4f174d0eef4a2bf85096
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/ofYN04WWtD60oeDP45pQN.jpeg
paper_authors.fullname: Simian Luo
paper_authors.type: user
paper_authors.user: SimianLuo
paper_authors._id: 659f529fd56cbade8db370ad
paper_authors.name: Enze Xie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-11T10:38:55.039Z
paper_authors._id: 616961bdd3f656f79ad18ec1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/616961bdd3f656f79ad18ec1/NYpxas378lDevRuUDepJJ.jpeg
paper_authors.fullname: xieenze
paper_authors.type: user
paper_authors.user: xieenze
paper_authors._id: 659f529fd56cbade8db370ae
paper_authors.name: Sayak Paul
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-11T08:13:18.092Z
paper_authors._id: 5f7fbd813e94f16a85448745
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1649681653581-5f7fbd813e94f16a85448745.jpeg
paper_authors.fullname: Sayak Paul
paper_authors.type: user
paper_authors.user: sayakpaul
paper_authors._id: 659f529fd56cbade8db370af
paper_authors.name: Ping Luo
paper_authors._id: 659f529fd56cbade8db370b0
paper_authors.name: Hang Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-11T10:35:02.476Z
paper_authors._id: 63bc8467b8c61b8aa49c3e11
paper_authors.avatarUrl: /avatars/f75a33b6bba1bb6aecbe018dcef39131.svg
paper_authors.fullname: Hang Zhao
paper_authors.type: user
paper_authors.user: sgzhaohang
paper_authors._id: 659f529fd56cbade8db370b1
paper_authors.name: Zhenguo Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-11T10:35:30.043Z
paper_authors._id: 639944bef7c4648254d124f0
paper_authors.avatarUrl: /avatars/c5cbb87cd51cc1341844f67d42c55151.svg
paper_authors.fullname: lzg
paper_authors.type: user
paper_authors.user: lizhenguo
paper_title: PIXART-δ: Fast and Controllable Image Generation with Latent
    Consistency Models
paper_summary: This technical report introduces PIXART-{\delta}, a text-to-image synthesis
  framework that integrates the Latent Consistency Model (LCM) and ControlNet
  into the advanced PIXART-{\alpha} model. PIXART-{\alpha} is recognized for its
  ability to generate high-quality images of 1024px resolution through a
  remarkably efficient training process. The integration of LCM in
  PIXART-{\delta} significantly accelerates the inference speed, enabling the
  production of high-quality images in just 2-4 steps. Notably, PIXART-{\delta}
  achieves a breakthrough 0.5 seconds for generating 1024x1024 pixel images,
  marking a 7x improvement over the PIXART-{\alpha}. Additionally,
  PIXART-{\delta} is designed to be efficiently trainable on 32GB V100 GPUs
  within a single day. With its 8-bit inference capability (von Platen et al.,
  2023), PIXART-{\delta} can synthesize 1024px images within 8GB GPU memory
  constraints, greatly enhancing its usability and accessibility. Furthermore,
  incorporating a ControlNet-like module enables fine-grained control over
  text-to-image diffusion models. We introduce a novel ControlNet-Transformer
  architecture, specifically tailored for Transformers, achieving explicit
  controllability alongside high-quality image generation. As a state-of-the-art,
  open-source image generation model, PIXART-{\delta} offers a promising
  alternative to the Stable Diffusion family of models, contributing
  significantly to text-to-image synthesis.
paper_ai_summary: PIXART-{\delta} integrates Latent Consistency Model (LCM) and ControlNet into PIXART-{\alpha} for accelerated, high-quality text-to-image synthesis with efficient training and inference.
paper_ai_keywords: Latent Consistency Model (LCM) ControlNet text-to-image synthesis diffusion models ControlNet-Transformer 8-bit inference