title: SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute
    in Linear Diffusion Transformer
summary: This paper presents SANA-1.5, a linear Diffusion Transformer for efficient
  scaling in text-to-image generation. Building upon SANA-1.0, we introduce three
  key innovations: (1) Efficient Training Scaling: A depth-growth paradigm that
  enables scaling from 1.6B to 4.8B parameters with significantly reduced
  computational resources, combined with a memory-efficient 8-bit optimizer. (2)
  Model Depth Pruning: A block importance analysis technique for efficient model
  compression to arbitrary sizes with minimal quality loss. (3) Inference-time
  Scaling: A repeated sampling strategy that trades computation for model
  capacity, enabling smaller models to match larger model quality at inference
  time. Through these strategies, SANA-1.5 achieves a text-image alignment score
  of 0.72 on GenEval, which can be further improved to 0.80 through inference
  scaling, establishing a new SoTA on GenEval benchmark. These innovations enable
  efficient model scaling across different compute budgets while maintaining high
  quality, making high-quality image generation more accessible.
paper_authors._id: 679d7fae1f4b90cfa7b74d0b
paper_authors.name: Enze Xie
paper_authors._id: 679d7fae1f4b90cfa7b74d0c
paper_authors.name: Junsong Chen
paper_authors._id: 679d7fae1f4b90cfa7b74d0d
paper_authors.name: Yuyang Zhao
paper_authors._id: 679d7fae1f4b90cfa7b74d0e
paper_authors.name: Jincheng Yu
paper_authors._id: 679d7fae1f4b90cfa7b74d0f
paper_authors.name: Ligeng Zhu
paper_authors._id: 679d7fae1f4b90cfa7b74d10
paper_authors.name: Yujun Lin
paper_authors._id: 679d7fae1f4b90cfa7b74d11
paper_authors.name: Zhekai Zhang
paper_authors._id: 679d7fae1f4b90cfa7b74d12
paper_authors.name: Muyang Li
paper_authors._id: 679d7fae1f4b90cfa7b74d13
paper_authors.name: Junyu Chen
paper_authors._id: 679d7fae1f4b90cfa7b74d14
paper_authors.name: Han Cai
paper_authors._id: 679d7fae1f4b90cfa7b74d15
paper_authors.name: Bingchen Liu
paper_authors._id: 679d7fae1f4b90cfa7b74d16
paper_authors.name: Daquan Zhou
paper_authors._id: 679d7fae1f4b90cfa7b74d17
paper_authors.name: Song Han
paper_title: SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute
    in Linear Diffusion Transformer
paper_summary: This paper presents SANA-1.5, a linear Diffusion Transformer for efficient
  scaling in text-to-image generation. Building upon SANA-1.0, we introduce three
  key innovations: (1) Efficient Training Scaling: A depth-growth paradigm that
  enables scaling from 1.6B to 4.8B parameters with significantly reduced
  computational resources, combined with a memory-efficient 8-bit optimizer. (2)
  Model Depth Pruning: A block importance analysis technique for efficient model
  compression to arbitrary sizes with minimal quality loss. (3) Inference-time
  Scaling: A repeated sampling strategy that trades computation for model
  capacity, enabling smaller models to match larger model quality at inference
  time. Through these strategies, SANA-1.5 achieves a text-image alignment score
  of 0.72 on GenEval, which can be further improved to 0.80 through inference
  scaling, establishing a new SoTA on GenEval benchmark. These innovations enable
  efficient model scaling across different compute budgets while maintaining high
  quality, making high-quality image generation more accessible.
paper_ai_keywords: Diffusion Transformer depth-growth paradigm memory-efficient 8-bit optimizer block importance analysis model depth pruning repeated sampling strategy text-image alignment score GenEval SoTA