SDXS: Real-Time One-Step Latent Diffusion Models with Image Conditions
Recent advancements in diffusion models have positioned them at the forefront
of image generation. Despite their superior performance, diffusion models are
not without drawbacks; they are characterized by complex architectures and
substantial computational demands, resulting in significant latency due to
their iterative sampling process. To mitigate these limitations, we introduce a
dual approach involving model miniaturization and a reduction in sampling
steps, aimed at significantly decreasing model latency. Our methodology
leverages knowledge distillation to streamline the U-Net and image decoder
architectures, and introduces an innovative one-step DM training technique that
utilizes feature matching and score distillation. We present two models,
SDXS-512 and SDXS-1024, achieving inference speeds of approximately 100 FPS
(30x faster than SD v1.5) and 30 FP (60x faster than SDXL) on a single GPU,
respectively. Moreover, our training approach offers promising applications in
image-conditioned control, facilitating efficient image-to-image translation.
66024548561089b1a7f80cad
Yuda Song
admin_assigned
2024-03-26T09:34:58.423Z
6264bf5a1ed8d81e47ae3a62
https://cdn-avatars.huggingface.co/v1/production/uploads/1650769741842-noauth.jpeg
Yuda Song
user
IDKiro
66024548561089b1a7f80cae
Zehao Sun
66024548561089b1a7f80caf
Xuanwu Yin
SDXS: Real-Time One-Step Latent Diffusion Models with Image Conditions
Recent advancements in diffusion models have positioned them at the forefront
of image generation. Despite their superior performance, diffusion models are
not without drawbacks; they are characterized by complex architectures and
substantial computational demands, resulting in significant latency due to
their iterative sampling process. To mitigate these limitations, we introduce a
dual approach involving model miniaturization and a reduction in sampling
steps, aimed at significantly decreasing model latency. Our methodology
leverages knowledge distillation to streamline the U-Net and image decoder
architectures, and introduces an innovative one-step DM training technique that
utilizes feature matching and score distillation. We present two models,
SDXS-512 and SDXS-1024, achieving inference speeds of approximately 100 FPS
(30x faster than SD v1.5) and 30 FP (60x faster than SDXL) on a single GPU,
respectively. Moreover, our training approach offers promising applications in
image-conditioned control, facilitating efficient image-to-image translation.
A dual approach using model miniaturization and reduced sampling steps improves diffusion models' inference speed through knowledge distillation and feature matching.
diffusion models U-Net image decoder knowledge distillation feature matching score distillation SDXS-512 SDXS-1024