title: Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image
    Synthesis
summary: Recently, a series of diffusion-aware distillation algorithms have emerged to
  alleviate the computational overhead associated with the multi-step inference
  process of Diffusion Models (DMs). Current distillation techniques often
  dichotomize into two distinct aspects: i) ODE Trajectory Preservation; and ii)
  ODE Trajectory Reformulation. However, these approaches suffer from severe
  performance degradation or domain shifts. To address these limitations, we
  propose Hyper-SD, a novel framework that synergistically amalgamates the
  advantages of ODE Trajectory Preservation and Reformulation, while maintaining
  near-lossless performance during step compression. Firstly, we introduce
  Trajectory Segmented Consistency Distillation to progressively perform
  consistent distillation within pre-defined time-step segments, which
  facilitates the preservation of the original ODE trajectory from a higher-order
  perspective. Secondly, we incorporate human feedback learning to boost the
  performance of the model in a low-step regime and mitigate the performance loss
  incurred by the distillation process. Thirdly, we integrate score distillation
  to further improve the low-step generation capability of the model and offer
  the first attempt to leverage a unified LoRA to support the inference process
  at all steps. Extensive experiments and user studies demonstrate that Hyper-SD
  achieves SOTA performance from 1 to 8 inference steps for both SDXL and SD1.5.
  For example, Hyper-SDXL surpasses SDXL-Lightning by +0.68 in CLIP Score and
  +0.51 in Aes Score in the 1-step inference.
paper_authors._id: 662717e588596f571aea8ec5
paper_authors.name: Yuxi Ren
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-23T08:04:10.414Z
paper_authors._id: 6618d5e83b412cdc85334ca8
paper_authors.avatarUrl: /avatars/5fe356d58c4c822a60370dbee8d78a69.svg
paper_authors.fullname: renyuxi
paper_authors.type: user
paper_authors.user: renyuxi
paper_authors._id: 662717e588596f571aea8ec6
paper_authors.name: Xin Xia
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-23T08:04:46.842Z
paper_authors._id: 63089a78ff78e2aead8d10e7
paper_authors.avatarUrl: /avatars/f326fd08abee7e31599a78923be30003.svg
paper_authors.fullname: XinXia
paper_authors.type: user
paper_authors.user: XiaXin-Aloys
paper_authors._id: 662717e588596f571aea8ec7
paper_authors.name: Yanzuo Lu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-23T07:13:59.809Z
paper_authors._id: 6614cbd40bbea65e71db4e1f
paper_authors.avatarUrl: /avatars/ca8ff74887bbf8eb3f5b04ae9bb6d05b.svg
paper_authors.fullname: Yanzuo Lu
paper_authors.type: user
paper_authors.user: oliveryanzuolu
paper_authors._id: 662717e588596f571aea8ec8
paper_authors.name: Jiacheng Zhang
paper_authors._id: 662717e588596f571aea8ec9
paper_authors.name: Jie Wu
paper_authors._id: 662717e588596f571aea8eca
paper_authors.name: Pan Xie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-23T08:05:25.748Z
paper_authors._id: 62d61f703bf5e059f7d24efd
paper_authors.avatarUrl: /avatars/b4a7651de5dfd8a1c26e9ca559886b26.svg
paper_authors.fullname: Xie Pan
paper_authors.type: user
paper_authors.user: panxie
paper_authors._id: 662717e588596f571aea8ecb
paper_authors.name: Xing Wang
paper_authors._id: 662717e588596f571aea8ecc
paper_authors.name: Xuefeng Xiao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-23T08:05:44.505Z
paper_authors._id: 646b7f71df2609a541c1ab9f
paper_authors.avatarUrl: /avatars/48b82e5fd9b06f41ff825507c36816cd.svg
paper_authors.fullname: Xuefeng Xiao
paper_authors.type: user
paper_authors.user: xiaoxuefeng
paper_title: Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image
    Synthesis
paper_summary: Recently, a series of diffusion-aware distillation algorithms have emerged to
  alleviate the computational overhead associated with the multi-step inference
  process of Diffusion Models (DMs). Current distillation techniques often
  dichotomize into two distinct aspects: i) ODE Trajectory Preservation; and ii)
  ODE Trajectory Reformulation. However, these approaches suffer from severe
  performance degradation or domain shifts. To address these limitations, we
  propose Hyper-SD, a novel framework that synergistically amalgamates the
  advantages of ODE Trajectory Preservation and Reformulation, while maintaining
  near-lossless performance during step compression. Firstly, we introduce
  Trajectory Segmented Consistency Distillation to progressively perform
  consistent distillation within pre-defined time-step segments, which
  facilitates the preservation of the original ODE trajectory from a higher-order
  perspective. Secondly, we incorporate human feedback learning to boost the
  performance of the model in a low-step regime and mitigate the performance loss
  incurred by the distillation process. Thirdly, we integrate score distillation
  to further improve the low-step generation capability of the model and offer
  the first attempt to leverage a unified LoRA to support the inference process
  at all steps. Extensive experiments and user studies demonstrate that Hyper-SD
  achieves SOTA performance from 1 to 8 inference steps for both SDXL and SD1.5.
  For example, Hyper-SDXL surpasses SDXL-Lightning by +0.68 in CLIP Score and
  +0.51 in Aes Score in the 1-step inference.
paper_ai_summary: Hyper-SD distills Diffusion Models by preserving and reformulating ODE trajectories, incorporating human feedback, and using score distillation, achieving state-of-the-art performance in step compression.
paper_ai_keywords: diffusion-aware distillation Diffusion Models ODE Trajectory Preservation ODE Trajectory Reformulation Hyper-SD Trajectory Segmented Consistency Distillation human feedback learning score distillation LoRA CLIP Score Aes Score SDXL SD1.5