title: SDXS: Real-Time One-Step Latent Diffusion Models with Image Conditions
summary: Recent advancements in diffusion models have positioned them at the forefront
  of image generation. Despite their superior performance, diffusion models are
  not without drawbacks; they are characterized by complex architectures and
  substantial computational demands, resulting in significant latency due to
  their iterative sampling process. To mitigate these limitations, we introduce a
  dual approach involving model miniaturization and a reduction in sampling
  steps, aimed at significantly decreasing model latency. Our methodology
  leverages knowledge distillation to streamline the U-Net and image decoder
  architectures, and introduces an innovative one-step DM training technique that
  utilizes feature matching and score distillation. We present two models,
  SDXS-512 and SDXS-1024, achieving inference speeds of approximately 100 FPS
  (30x faster than SD v1.5) and 30 FP (60x faster than SDXL) on a single GPU,
  respectively. Moreover, our training approach offers promising applications in
  image-conditioned control, facilitating efficient image-to-image translation.
paper_authors._id: 66024548561089b1a7f80cad
paper_authors.name: Yuda Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-26T09:34:58.423Z
paper_authors._id: 6264bf5a1ed8d81e47ae3a62
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1650769741842-noauth.jpeg
paper_authors.fullname: Yuda Song
paper_authors.type: user
paper_authors.user: IDKiro
paper_authors._id: 66024548561089b1a7f80cae
paper_authors.name: Zehao Sun
paper_authors._id: 66024548561089b1a7f80caf
paper_authors.name: Xuanwu Yin
paper_title: SDXS: Real-Time One-Step Latent Diffusion Models with Image Conditions
paper_summary: Recent advancements in diffusion models have positioned them at the forefront
  of image generation. Despite their superior performance, diffusion models are
  not without drawbacks; they are characterized by complex architectures and
  substantial computational demands, resulting in significant latency due to
  their iterative sampling process. To mitigate these limitations, we introduce a
  dual approach involving model miniaturization and a reduction in sampling
  steps, aimed at significantly decreasing model latency. Our methodology
  leverages knowledge distillation to streamline the U-Net and image decoder
  architectures, and introduces an innovative one-step DM training technique that
  utilizes feature matching and score distillation. We present two models,
  SDXS-512 and SDXS-1024, achieving inference speeds of approximately 100 FPS
  (30x faster than SD v1.5) and 30 FP (60x faster than SDXL) on a single GPU,
  respectively. Moreover, our training approach offers promising applications in
  image-conditioned control, facilitating efficient image-to-image translation.
paper_ai_summary: A dual approach using model miniaturization and reduced sampling steps improves diffusion models' inference speed through knowledge distillation and feature matching.
paper_ai_keywords: diffusion models U-Net image decoder knowledge distillation feature matching score distillation SDXS-512 SDXS-1024