title: Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model
summary: As a fundamental backbone for video generation, diffusion models are
  challenged by low inference speed due to the sequential nature of denoising.
  Previous methods speed up the models by caching and reusing model outputs at
  uniformly selected timesteps. However, such a strategy neglects the fact that
  differences among model outputs are not uniform across timesteps, which hinders
  selecting the appropriate model outputs to cache, leading to a poor balance
  between inference efficiency and visual quality. In this study, we introduce
  Timestep Embedding Aware Cache (TeaCache), a training-free caching approach
  that estimates and leverages the fluctuating differences among model outputs
  across timesteps. Rather than directly using the time-consuming model outputs,
  TeaCache focuses on model inputs, which have a strong correlation with the
  modeloutputs while incurring negligible computational cost. TeaCache first
  modulates the noisy inputs using the timestep embeddings to ensure their
  differences better approximating those of model outputs. TeaCache then
  introduces a rescaling strategy to refine the estimated differences and
  utilizes them to indicate output caching. Experiments show that TeaCache
  achieves up to 4.41x acceleration over Open-Sora-Plan with negligible (-0.07%
  Vbench score) degradation of visual quality.
paper_authors._id: 674d371b5e924d18e56a450c
paper_authors.name: Feng Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-02T09:34:03.626Z
paper_authors._id: 661658b9e398e4f0757221ce
paper_authors.avatarUrl: /avatars/9e9d55145661d00e2a9e029ba2e085d8.svg
paper_authors.fullname: Liu Feng
paper_authors.type: user
paper_authors.user: LiewFeng
paper_authors._id: 674d371b5e924d18e56a450d
paper_authors.name: Shiwei Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-02T09:11:39.991Z
paper_authors._id: 63a1200045edac9f7508bae9
paper_authors.avatarUrl: /avatars/e84f0b045f32c5b8b4da43458650b925.svg
paper_authors.fullname: Shiwei Zhang
paper_authors.type: user
paper_authors.user: StevenZhang
paper_authors._id: 674d371b5e924d18e56a450e
paper_authors.name: Xiaofeng Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-02T08:51:21.743Z
paper_authors._id: 6426616ea5ec4a5cbc535634
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6426616ea5ec4a5cbc535634/hH6JsxnXeakH3mBTeNNmO.jpeg
paper_authors.fullname: JeffWang
paper_authors.type: user
paper_authors.user: Jeff-Wang
paper_authors._id: 674d371b5e924d18e56a450f
paper_authors.name: Yujie Wei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-02T09:12:00.442Z
paper_authors._id: 637f70d6fab5db9101c3dfc8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/637f70d6fab5db9101c3dfc8/NgkYNXWLDavLbrnCby2Fl.jpeg
paper_authors.fullname: Yujie Wei
paper_authors.type: user
paper_authors.user: weilllllls
paper_authors._id: 674d371b5e924d18e56a4510
paper_authors.name: Haonan Qiu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-02T09:12:05.919Z
paper_authors._id: 63023b6ab002e9a4a2152890
paper_authors.avatarUrl: /avatars/cae8ba0a8d61fb4e576934431f43991b.svg
paper_authors.fullname: Haonan Qiu
paper_authors.type: user
paper_authors.user: MoonQiu
paper_authors._id: 674d371b5e924d18e56a4511
paper_authors.name: Yuzhong Zhao
paper_authors._id: 674d371b5e924d18e56a4512
paper_authors.name: Yingya Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-02T09:34:01.699Z
paper_authors._id: 66f3dd5c0fc8cc6ae16af8d5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66f3dd5c0fc8cc6ae16af8d5/xd-PuVvj5Ber9yvAFm3jq.jpeg
paper_authors.fullname: voidoc
paper_authors.type: user
paper_authors.user: ldmLDM77
paper_authors._id: 674d371b5e924d18e56a4513
paper_authors.name: Qixiang Ye
paper_authors._id: 674d371b5e924d18e56a4514
paper_authors.name: Fang Wan
paper_title: Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model
paper_summary: As a fundamental backbone for video generation, diffusion models are
  challenged by low inference speed due to the sequential nature of denoising.
  Previous methods speed up the models by caching and reusing model outputs at
  uniformly selected timesteps. However, such a strategy neglects the fact that
  differences among model outputs are not uniform across timesteps, which hinders
  selecting the appropriate model outputs to cache, leading to a poor balance
  between inference efficiency and visual quality. In this study, we introduce
  Timestep Embedding Aware Cache (TeaCache), a training-free caching approach
  that estimates and leverages the fluctuating differences among model outputs
  across timesteps. Rather than directly using the time-consuming model outputs,
  TeaCache focuses on model inputs, which have a strong correlation with the
  modeloutputs while incurring negligible computational cost. TeaCache first
  modulates the noisy inputs using the timestep embeddings to ensure their
  differences better approximating those of model outputs. TeaCache then
  introduces a rescaling strategy to refine the estimated differences and
  utilizes them to indicate output caching. Experiments show that TeaCache
  achieves up to 4.41x acceleration over Open-Sora-Plan with negligible (-0.07%
  Vbench score) degradation of visual quality.
paper_ai_keywords: diffusion models denoising timestep embeddings caching rescaling strategy Vbench score