Training-Free Efficient Video Generation via Dynamic Token Carving
Despite the remarkable generation quality of video Diffusion Transformer
(DiT) models, their practical deployment is severely hindered by extensive
computational requirements. This inefficiency stems from two key challenges:
the quadratic complexity of self-attention with respect to token length and the
multi-step nature of diffusion models. To address these limitations, we present
Jenga, a novel inference pipeline that combines dynamic attention carving with
progressive resolution generation. Our approach leverages two key insights: (1)
early denoising steps do not require high-resolution latents, and (2) later
steps do not require dense attention. Jenga introduces a block-wise attention
mechanism that dynamically selects relevant token interactions using 3D
space-filling curves, alongside a progressive resolution strategy that
gradually increases latent resolution during generation. Experimental results
demonstrate that Jenga achieves substantial speedups across multiple
state-of-the-art video diffusion models while maintaining comparable generation
quality (8.83times speedup with 0.01\% performance drop on VBench). As a
plug-and-play solution, Jenga enables practical, high-quality video generation
on modern hardware by reducing inference time from minutes to seconds --
without requiring model retraining. Code:
https://github.com/dvlab-research/Jenga
682fe14abafb480b9595da32
Yuechen Zhang
claimed_verified
2025-05-26T08:16:29.629Z
6418554a0956be7233a1023e
https://cdn-avatars.huggingface.co/v1/production/uploads/6418554a0956be7233a1023e/9EKN0GoOpcDbvBDmAQEJf.png
zhang yuechen
user
julianjuaner
682fe14abafb480b9595da33
Jinbo Xing
claimed_verified
2025-05-26T08:16:27.131Z
64770e86d7cf39f2e937ae9a
https://cdn-avatars.huggingface.co/v1/production/uploads/64770e86d7cf39f2e937ae9a/pLqGg2z1KzQxCGpMwds-9.jpeg
Jinbo Xing
user
Doubiiu
682fe14abafb480b9595da34
Bin Xia
682fe14abafb480b9595da35
Shaoteng Liu
682fe14abafb480b9595da36
Bohao Peng
682fe14abafb480b9595da37
Xin Tao
682fe14abafb480b9595da38
Pengfei Wan
682fe14abafb480b9595da39
Eric Lo
682fe14abafb480b9595da3a
Jiaya Jia
Training-Free Efficient Video Generation via Dynamic Token Carving
Despite the remarkable generation quality of video Diffusion Transformer
(DiT) models, their practical deployment is severely hindered by extensive
computational requirements. This inefficiency stems from two key challenges:
the quadratic complexity of self-attention with respect to token length and the
multi-step nature of diffusion models. To address these limitations, we present
Jenga, a novel inference pipeline that combines dynamic attention carving with
progressive resolution generation. Our approach leverages two key insights: (1)
early denoising steps do not require high-resolution latents, and (2) later
steps do not require dense attention. Jenga introduces a block-wise attention
mechanism that dynamically selects relevant token interactions using 3D
space-filling curves, alongside a progressive resolution strategy that
gradually increases latent resolution during generation. Experimental results
demonstrate that Jenga achieves substantial speedups across multiple
state-of-the-art video diffusion models while maintaining comparable generation
quality (8.83times speedup with 0.01\% performance drop on VBench). As a
plug-and-play solution, Jenga enables practical, high-quality video generation
on modern hardware by reducing inference time from minutes to seconds --
without requiring model retraining. Code:
https://github.com/dvlab-research/Jenga
Jenga, a novel inference pipeline for video Diffusion Transformer models, combines dynamic attention carving and progressive resolution generation to significantly speed up video generation while maintaining high quality.
video Diffusion Transformer (DiT) self-attention diffusion models dynamic attention carving progressive resolution generation block-wise attention 3D space-filling curves