title: LightGen: Efficient Image Generation through Knowledge Distillation and
    Direct Preference Optimization
summary: Recent advances in text-to-image generation have primarily relied on
  extensive datasets and parameter-heavy architectures. These requirements
  severely limit accessibility for researchers and practitioners who lack
  substantial computational resources. In this paper, we introduce \model, an
  efficient training paradigm for image generation models that uses knowledge
  distillation (KD) and Direct Preference Optimization (DPO). Drawing inspiration
  from the success of data KD techniques widely adopted in Multi-Modal Large
  Language Models (MLLMs), LightGen distills knowledge from state-of-the-art
  (SOTA) text-to-image models into a compact Masked Autoregressive (MAR)
  architecture with only 0.7B parameters. Using a compact synthetic dataset of
  just 2M high-quality images generated from varied captions, we demonstrate
  that data diversity significantly outweighs data volume in determining model
  performance. This strategy dramatically reduces computational demands and
  reduces pre-training time from potentially thousands of GPU-days to merely 88
  GPU-days. Furthermore, to address the inherent shortcomings of synthetic data,
  particularly poor high-frequency details and spatial inaccuracies, we integrate
  the DPO technique that refines image fidelity and positional accuracy.
  Comprehensive experiments confirm that LightGen achieves image generation
  quality comparable to SOTA models while significantly reducing computational
  resources and expanding accessibility for resource-constrained environments.
  Code is available at https://github.com/XianfengWu01/LightGen
paper_authors._id: 67d0eb9cec69694dca382208
paper_authors.name: Xianfeng Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:48:54.455Z
paper_authors._id: 6581a9e2e4bcbca0322e3608
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/IxvVmIg2YQaqa0ZEV6JPa.png
paper_authors.fullname: Xianfeng Wu
paper_authors.type: user
paper_authors.user: Beckham808
paper_authors._id: 67d0eb9cec69694dca382209
paper_authors.name: Yajing Bai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:49:04.779Z
paper_authors._id: 66e15e6a9cbd604971999bf2
paper_authors.avatarUrl: /avatars/dc02874bbe33e33c9f0dad4b40e55e01.svg
paper_authors.fullname: Yajing Bai
paper_authors.type: user
paper_authors.user: YajingB
paper_authors._id: 67d0eb9cec69694dca38220a
paper_authors.name: Haoze Zheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:49:13.485Z
paper_authors._id: 662a8cc138c02c106f883183
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/662a8cc138c02c106f883183/FvWqqawwAPA-aEGoPt6i5.jpeg
paper_authors.fullname: Haoze Zheng
paper_authors.type: user
paper_authors.user: Zanghu-Ze
paper_authors._id: 67d0eb9cec69694dca38220b
paper_authors.name: Harold Haodong Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:49:27.804Z
paper_authors._id: 6570450a78d7aca0c361a177
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6570450a78d7aca0c361a177/z0GrnXEsjK2_G-hFfQhKv.jpeg
paper_authors.fullname: Harold Chen
paper_authors.type: user
paper_authors.user: Harold328
paper_authors._id: 67d0eb9cec69694dca38220c
paper_authors.name: Yexin Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:49:38.339Z
paper_authors._id: 6497ff395b5d43c1c77d2a11
paper_authors.avatarUrl: /avatars/e3704701ed3ea0c1c165001521f40086.svg
paper_authors.fullname: Yexin Liu
paper_authors.type: user
paper_authors.user: AI4VR
paper_authors._id: 67d0eb9cec69694dca38220d
paper_authors.name: Zihao Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:49:48.448Z
paper_authors._id: 6757a1fde1a2dd778051b9e2
paper_authors.avatarUrl: /avatars/bd2b5bb509902e1ad88a4b63e6cd0187.svg
paper_authors.fullname: Zihao Wang
paper_authors.type: user
paper_authors.user: ZihaoWang
paper_authors._id: 67d0eb9cec69694dca38220e
paper_authors.name: Xuran Ma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:49:56.503Z
paper_authors._id: 6669bd8241fcd2f39a15a18b
paper_authors.avatarUrl: /avatars/1b1f3cfbc5d66030810eb071f2ecd57c.svg
paper_authors.fullname: Xuran Ma
paper_authors.type: user
paper_authors.user: lemonnnn
paper_authors._id: 67d0eb9cec69694dca38220f
paper_authors.name: Wen-Jie Shu
paper_authors._id: 67d0eb9cec69694dca382210
paper_authors.name: Xianzu Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:50:16.096Z
paper_authors._id: 66dfadac9f6980cf98570efa
paper_authors.avatarUrl: /avatars/7d1035ed163e2c0a581771b1c723a2d4.svg
paper_authors.fullname: Xianzu Wu
paper_authors.type: user
paper_authors.user: Xianzu
paper_authors._id: 67d0eb9cec69694dca382211
paper_authors.name: Harry Yang
paper_authors._id: 67d0eb9cec69694dca382212
paper_authors.name: Ser-Nam Lim
paper_title: LightGen: Efficient Image Generation through Knowledge Distillation and
    Direct Preference Optimization
paper_summary: Recent advances in text-to-image generation have primarily relied on
  extensive datasets and parameter-heavy architectures. These requirements
  severely limit accessibility for researchers and practitioners who lack
  substantial computational resources. In this paper, we introduce \model, an
  efficient training paradigm for image generation models that uses knowledge
  distillation (KD) and Direct Preference Optimization (DPO). Drawing inspiration
  from the success of data KD techniques widely adopted in Multi-Modal Large
  Language Models (MLLMs), LightGen distills knowledge from state-of-the-art
  (SOTA) text-to-image models into a compact Masked Autoregressive (MAR)
  architecture with only 0.7B parameters. Using a compact synthetic dataset of
  just 2M high-quality images generated from varied captions, we demonstrate
  that data diversity significantly outweighs data volume in determining model
  performance. This strategy dramatically reduces computational demands and
  reduces pre-training time from potentially thousands of GPU-days to merely 88
  GPU-days. Furthermore, to address the inherent shortcomings of synthetic data,
  particularly poor high-frequency details and spatial inaccuracies, we integrate
  the DPO technique that refines image fidelity and positional accuracy.
  Comprehensive experiments confirm that LightGen achieves image generation
  quality comparable to SOTA models while significantly reducing computational
  resources and expanding accessibility for resource-constrained environments.
  Code is available at https://github.com/XianfengWu01/LightGen
paper_ai_summary: LightGen reduces computational resource requirements in text-to-image generation by using knowledge distillation and direct preference optimization, making state-of-the-art performance accessible.
paper_ai_keywords: knowledge distillation Direct Preference Optimization Masked Autoregressive synthetic dataset high-frequency details spatial inaccuracies image fidelity positional accuracy