title: PanGu-Draw: Advancing Resource-Efficient Text-to-Image Synthesis with
    Time-Decoupled Training and Reusable Coop-Diffusion
summary: Current large-scale diffusion models represent a giant leap forward in
  conditional image synthesis, capable of interpreting diverse cues like text,
  human poses, and edges. However, their reliance on substantial computational
  resources and extensive data collection remains a bottleneck. On the other
  hand, the integration of existing diffusion models, each specialized for
  different controls and operating in unique latent spaces, poses a challenge due
  to incompatible image resolutions and latent space embedding structures,
  hindering their joint use. Addressing these constraints, we present
  "PanGu-Draw", a novel latent diffusion model designed for resource-efficient
  text-to-image synthesis that adeptly accommodates multiple control signals. We
  first propose a resource-efficient Time-Decoupling Training Strategy, which
  splits the monolithic text-to-image model into structure and texture
  generators. Each generator is trained using a regimen that maximizes data
  utilization and computational efficiency, cutting data preparation by 48% and
  reducing training resources by 51%. Secondly, we introduce "Coop-Diffusion", an
  algorithm that enables the cooperative use of various pre-trained diffusion
  models with different latent spaces and predefined resolutions within a unified
  denoising process. This allows for multi-control image synthesis at arbitrary
  resolutions without the necessity for additional data or retraining. Empirical
  validations of Pangu-Draw show its exceptional prowess in text-to-image and
  multi-control image generation, suggesting a promising direction for future
  model training efficiencies and generation versatility. The largest 5B T2I
  PanGu-Draw model is released on the Ascend platform. Project page:
  https://pangu-draw.github.io
paper_authors._id: 658e3c267fe0235473d341a9
paper_authors.name: Guansong Lu
paper_authors._id: 658e3c267fe0235473d341aa
paper_authors.name: Yuanfan Guo
paper_authors._id: 658e3c267fe0235473d341ab
paper_authors.name: Jianhua Han
paper_authors._id: 658e3c267fe0235473d341ac
paper_authors.name: Minzhe Niu
paper_authors._id: 658e3c267fe0235473d341ad
paper_authors.name: Yihan Zeng
paper_authors._id: 658e3c267fe0235473d341ae
paper_authors.name: Songcen Xu
paper_authors._id: 658e3c267fe0235473d341af
paper_authors.name: Zeyi Huang
paper_authors._id: 658e3c267fe0235473d341b0
paper_authors.name: Zhao Zhong
paper_authors._id: 658e3c267fe0235473d341b1
paper_authors.name: Wei Zhang
paper_authors._id: 658e3c267fe0235473d341b2
paper_authors.name: Hang Xu
paper_title: PanGu-Draw: Advancing Resource-Efficient Text-to-Image Synthesis with
    Time-Decoupled Training and Reusable Coop-Diffusion
paper_summary: Current large-scale diffusion models represent a giant leap forward in
  conditional image synthesis, capable of interpreting diverse cues like text,
  human poses, and edges. However, their reliance on substantial computational
  resources and extensive data collection remains a bottleneck. On the other
  hand, the integration of existing diffusion models, each specialized for
  different controls and operating in unique latent spaces, poses a challenge due
  to incompatible image resolutions and latent space embedding structures,
  hindering their joint use. Addressing these constraints, we present
  "PanGu-Draw", a novel latent diffusion model designed for resource-efficient
  text-to-image synthesis that adeptly accommodates multiple control signals. We
  first propose a resource-efficient Time-Decoupling Training Strategy, which
  splits the monolithic text-to-image model into structure and texture
  generators. Each generator is trained using a regimen that maximizes data
  utilization and computational efficiency, cutting data preparation by 48% and
  reducing training resources by 51%. Secondly, we introduce "Coop-Diffusion", an
  algorithm that enables the cooperative use of various pre-trained diffusion
  models with different latent spaces and predefined resolutions within a unified
  denoising process. This allows for multi-control image synthesis at arbitrary
  resolutions without the necessity for additional data or retraining. Empirical
  validations of Pangu-Draw show its exceptional prowess in text-to-image and
  multi-control image generation, suggesting a promising direction for future
  model training efficiencies and generation versatility. The largest 5B T2I
  PanGu-Draw model is released on the Ascend platform. Project page:
  https://pangu-draw.github.io
paper_ai_summary: PanGu-Draw, a latent diffusion model, enhances resource efficiency in text-to-image synthesis by integrating variable control signals through time-decoupling and cooperative diffusion techniques.
paper_ai_keywords: latent diffusion model Time-Decoupling Training Strategy structure generator texture generator Coop-Diffusion pre-trained diffusion models variable control signals unified denoising process text-to-image multi-control image generation