Progressive Knowledge Distillation Of Stable Diffusion XL Using Layer
  Level Loss
Stable Diffusion XL (SDXL) has become the best open source text-to-image
model (T2I) for its versatility and top-notch image quality. Efficiently
addressing the computational demands of SDXL models is crucial for wider reach
and applicability. In this work, we introduce two scaled-down variants, Segmind
Stable Diffusion (SSD-1B) and Segmind-Vega, with 1.3B and 0.74B parameter
UNets, respectively, achieved through progressive removal using layer-level
losses focusing on reducing the model size while preserving generative quality.
We release these models weights at https://hf.co/Segmind. Our methodology
involves the elimination of residual networks and transformer blocks from the
U-Net structure of SDXL, resulting in significant reductions in parameters, and
latency. Our compact models effectively emulate the original SDXL by
capitalizing on transferred knowledge, achieving competitive results against
larger multi-billion parameter SDXL. Our work underscores the efficacy of
knowledge distillation coupled with layer-level losses in reducing model size
while preserving the high-quality generative capabilities of SDXL, thus
facilitating more accessible deployment in resource-constrained environments.
659b635821a74316431e14e8
Yatharth Gupta
claimed_verified
2024-01-08T08:47:26.552Z
62f8ca074588fe31f4361dae
https://cdn-avatars.huggingface.co/v1/production/uploads/62f8ca074588fe31f4361dae/F2k343TPD7KVfW3P26IRs.jpeg
Yatharth Gupta
user
Warlord-K
659b635821a74316431e14e9
Vishnu V. Jaddipal
claimed_verified
2024-01-08T08:47:28.507Z
639362039694ec0d025f194d
https://cdn-avatars.huggingface.co/v1/production/uploads/639362039694ec0d025f194d/jmQetjDK3ISH_IQb7SBGC.png
Vishnu V Jaddipal
user
Icar
659b635821a74316431e14ea
Harish Prabhala
extracted_confirmed
2024-01-08T04:13:21.785Z
6412b77da26724794c3c82b5
/avatars/3f20c2698b29462f1e5231bbaa51478b.svg
Harish Segmind
user
harishseg
659b635821a74316431e14eb
Sayak Paul
extracted_confirmed
2024-01-08T07:39:54.480Z
5f7fbd813e94f16a85448745
https://cdn-avatars.huggingface.co/v1/production/uploads/1649681653581-5f7fbd813e94f16a85448745.jpeg
Sayak Paul
user
sayakpaul
659b635821a74316431e14ec
Patrick Von Platen
extracted_pending
2024-01-08T02:52:12.391Z
5dfcb1aada6d0311fd3d5448
https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg
Patrick von Platen
user
patrickvonplaten
Progressive Knowledge Distillation Of Stable Diffusion XL Using Layer
  Level Loss
Stable Diffusion XL (SDXL) has become the best open source text-to-image
model (T2I) for its versatility and top-notch image quality. Efficiently
addressing the computational demands of SDXL models is crucial for wider reach
and applicability. In this work, we introduce two scaled-down variants, Segmind
Stable Diffusion (SSD-1B) and Segmind-Vega, with 1.3B and 0.74B parameter
UNets, respectively, achieved through progressive removal using layer-level
losses focusing on reducing the model size while preserving generative quality.
We release these models weights at https://hf.co/Segmind. Our methodology
involves the elimination of residual networks and transformer blocks from the
U-Net structure of SDXL, resulting in significant reductions in parameters, and
latency. Our compact models effectively emulate the original SDXL by
capitalizing on transferred knowledge, achieving competitive results against
larger multi-billion parameter SDXL. Our work underscores the efficacy of
knowledge distillation coupled with layer-level losses in reducing model size
while preserving the high-quality generative capabilities of SDXL, thus
facilitating more accessible deployment in resource-constrained environments.
Two scaled-down variants of Stable Diffusion XL (SDXL), achieved through layer-level loss-driven pruning, maintain competitive image quality with reduced computational demands.
Stable Diffusion XL text-to-image T2I parameter-efficient U-Net residual networks transformer blocks knowledge distillation layer-level losses generative quality resource-constrained environments