FasterCache: Training-Free Video Diffusion Model Acceleration with High
  Quality
In this paper, we present \textit{FasterCache}, a novel
training-free strategy designed to accelerate the inference of video diffusion
models with high-quality generation. By analyzing existing cache-based methods,
we observe that directly reusing adjacent-step features degrades video
quality due to the loss of subtle variations. We further perform a pioneering
investigation of the acceleration potential of classifier-free guidance (CFG)
and reveal significant redundancy between conditional and unconditional
features within the same timestep. Capitalizing on these observations, we
introduce FasterCache to substantially accelerate diffusion-based video
generation. Our key contributions include a dynamic feature reuse strategy that
preserves both feature distinction and temporal continuity, and CFG-Cache which
optimizes the reuse of conditional and unconditional outputs to further enhance
inference speed without compromising video quality. We empirically evaluate
FasterCache on recent video diffusion models. Experimental results show that
FasterCache can significantly accelerate video generation (\eg 1.67times
speedup on Vchitect-2.0) while keeping video quality comparable to the
baseline, and consistently outperform existing methods in both inference speed
and video quality.
671f085f943be9e8dfcd3e6b
Zhengyao Lv
admin_assigned
2024-10-28T15:22:46.709Z
63243004dedcbcf92830e76b
https://cdn-avatars.huggingface.co/v1/production/uploads/63243004dedcbcf92830e76b/Ftv8UFhHQFMi3utTCoDVF.png
Zhengyao Lu
user
zlu
671f085f943be9e8dfcd3e6c
Chenyang Si
admin_assigned
2024-10-28T15:24:46.068Z
635f8ed47c05eb9f59963d3a
https://cdn-avatars.huggingface.co/v1/production/uploads/635f8ed47c05eb9f59963d3a/uQf4p9N9pSaFy87Wg9v4k.jpeg
ChenyangSi
user
ChenyangSi
671f085f943be9e8dfcd3e6d
Junhao Song
admin_assigned
2024-10-28T15:24:38.601Z
653a88011f981e43044ccd7d
/avatars/d7aaa588ab1b2491169523666e77d8a9.svg
Junhao
user
JunhaoSong
671f085f943be9e8dfcd3e6e
Zhenyu Yang
671f085f943be9e8dfcd3e6f
Yu Qiao
671f085f943be9e8dfcd3e70
Ziwei Liu
admin_assigned
2024-10-28T15:23:53.295Z
62ab1ac1d48b4d8b048a3473
https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
Ziwei Liu
user
liuziwei7
671f085f943be9e8dfcd3e71
Kwan-Yee K. Wong
FasterCache: Training-Free Video Diffusion Model Acceleration with High
  Quality
In this paper, we present \textit{FasterCache}, a novel
training-free strategy designed to accelerate the inference of video diffusion
models with high-quality generation. By analyzing existing cache-based methods,
we observe that directly reusing adjacent-step features degrades video
quality due to the loss of subtle variations. We further perform a pioneering
investigation of the acceleration potential of classifier-free guidance (CFG)
and reveal significant redundancy between conditional and unconditional
features within the same timestep. Capitalizing on these observations, we
introduce FasterCache to substantially accelerate diffusion-based video
generation. Our key contributions include a dynamic feature reuse strategy that
preserves both feature distinction and temporal continuity, and CFG-Cache which
optimizes the reuse of conditional and unconditional outputs to further enhance
inference speed without compromising video quality. We empirically evaluate
FasterCache on recent video diffusion models. Experimental results show that
FasterCache can significantly accelerate video generation (\eg 1.67times
speedup on Vchitect-2.0) while keeping video quality comparable to the
baseline, and consistently outperform existing methods in both inference speed
and video quality.
FasterCache, a training-free strategy, accelerates video diffusion model inference by dynamically reusing features while maintaining video quality.
video diffusion models feature reuse classifier-free guidance (CFG) dynamic feature reuse strategy temporal continuity Vchitect-2.0