LightGen: Efficient Image Generation through Knowledge Distillation and
  Direct Preference Optimization
Recent advances in text-to-image generation have primarily relied on
extensive datasets and parameter-heavy architectures. These requirements
severely limit accessibility for researchers and practitioners who lack
substantial computational resources. In this paper, we introduce \model, an
efficient training paradigm for image generation models that uses knowledge
distillation (KD) and Direct Preference Optimization (DPO). Drawing inspiration
from the success of data KD techniques widely adopted in Multi-Modal Large
Language Models (MLLMs), LightGen distills knowledge from state-of-the-art
(SOTA) text-to-image models into a compact Masked Autoregressive (MAR)
architecture with only 0.7B parameters. Using a compact synthetic dataset of
just 2M high-quality images generated from varied captions, we demonstrate
that data diversity significantly outweighs data volume in determining model
performance. This strategy dramatically reduces computational demands and
reduces pre-training time from potentially thousands of GPU-days to merely 88
GPU-days. Furthermore, to address the inherent shortcomings of synthetic data,
particularly poor high-frequency details and spatial inaccuracies, we integrate
the DPO technique that refines image fidelity and positional accuracy.
Comprehensive experiments confirm that LightGen achieves image generation
quality comparable to SOTA models while significantly reducing computational
resources and expanding accessibility for resource-constrained environments.
Code is available at https://github.com/XianfengWu01/LightGen
67d0eb9cec69694dca382208
Xianfeng Wu
admin_assigned
2025-03-12T15:48:54.455Z
6581a9e2e4bcbca0322e3608
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/IxvVmIg2YQaqa0ZEV6JPa.png
Xianfeng Wu
user
Beckham808
67d0eb9cec69694dca382209
Yajing Bai
admin_assigned
2025-03-12T15:49:04.779Z
66e15e6a9cbd604971999bf2
/avatars/dc02874bbe33e33c9f0dad4b40e55e01.svg
Yajing Bai
user
YajingB
67d0eb9cec69694dca38220a
Haoze Zheng
admin_assigned
2025-03-12T15:49:13.485Z
662a8cc138c02c106f883183
https://cdn-avatars.huggingface.co/v1/production/uploads/662a8cc138c02c106f883183/FvWqqawwAPA-aEGoPt6i5.jpeg
Haoze Zheng
user
Zanghu-Ze
67d0eb9cec69694dca38220b
Harold Haodong Chen
admin_assigned
2025-03-12T15:49:27.804Z
6570450a78d7aca0c361a177
https://cdn-avatars.huggingface.co/v1/production/uploads/6570450a78d7aca0c361a177/z0GrnXEsjK2_G-hFfQhKv.jpeg
Harold Chen
user
Harold328
67d0eb9cec69694dca38220c
Yexin Liu
admin_assigned
2025-03-12T15:49:38.339Z
6497ff395b5d43c1c77d2a11
/avatars/e3704701ed3ea0c1c165001521f40086.svg
Yexin Liu
user
AI4VR
67d0eb9cec69694dca38220d
Zihao Wang
admin_assigned
2025-03-12T15:49:48.448Z
6757a1fde1a2dd778051b9e2
/avatars/bd2b5bb509902e1ad88a4b63e6cd0187.svg
Zihao Wang
user
ZihaoWang
67d0eb9cec69694dca38220e
Xuran Ma
admin_assigned
2025-03-12T15:49:56.503Z
6669bd8241fcd2f39a15a18b
/avatars/1b1f3cfbc5d66030810eb071f2ecd57c.svg
Xuran Ma
user
lemonnnn
67d0eb9cec69694dca38220f
Wen-Jie Shu
67d0eb9cec69694dca382210
Xianzu Wu
admin_assigned
2025-03-12T15:50:16.096Z
66dfadac9f6980cf98570efa
/avatars/7d1035ed163e2c0a581771b1c723a2d4.svg
Xianzu Wu
user
Xianzu
67d0eb9cec69694dca382211
Harry Yang
67d0eb9cec69694dca382212
Ser-Nam Lim
LightGen: Efficient Image Generation through Knowledge Distillation and
  Direct Preference Optimization
Recent advances in text-to-image generation have primarily relied on
extensive datasets and parameter-heavy architectures. These requirements
severely limit accessibility for researchers and practitioners who lack
substantial computational resources. In this paper, we introduce \model, an
efficient training paradigm for image generation models that uses knowledge
distillation (KD) and Direct Preference Optimization (DPO). Drawing inspiration
from the success of data KD techniques widely adopted in Multi-Modal Large
Language Models (MLLMs), LightGen distills knowledge from state-of-the-art
(SOTA) text-to-image models into a compact Masked Autoregressive (MAR)
architecture with only 0.7B parameters. Using a compact synthetic dataset of
just 2M high-quality images generated from varied captions, we demonstrate
that data diversity significantly outweighs data volume in determining model
performance. This strategy dramatically reduces computational demands and
reduces pre-training time from potentially thousands of GPU-days to merely 88
GPU-days. Furthermore, to address the inherent shortcomings of synthetic data,
particularly poor high-frequency details and spatial inaccuracies, we integrate
the DPO technique that refines image fidelity and positional accuracy.
Comprehensive experiments confirm that LightGen achieves image generation
quality comparable to SOTA models while significantly reducing computational
resources and expanding accessibility for resource-constrained environments.
Code is available at https://github.com/XianfengWu01/LightGen
LightGen reduces computational resource requirements in text-to-image generation by using knowledge distillation and direct preference optimization, making state-of-the-art performance accessible.
knowledge distillation Direct Preference Optimization Masked Autoregressive synthetic dataset high-frequency details spatial inaccuracies image fidelity positional accuracy