Progressive Autoregressive Video Diffusion Models
Current frontier video diffusion models have demonstrated remarkable results
at generating high-quality videos. However, they can only generate short video
clips, normally around 10 seconds or 240 frames, due to computation limitations
during training. In this work, we show that existing models can be naturally
extended to autoregressive video diffusion models without changing the
architectures. Our key idea is to assign the latent frames with progressively
increasing noise levels rather than a single noise level, which allows for
fine-grained condition among the latents and large overlaps between the
attention windows. Such progressive video denoising allows our models to
autoregressively generate video frames without quality degradation or abrupt
scene changes. We present state-of-the-art results on long video generation at
1 minute (1440 frames at 24 FPS). Videos from this paper are available at
https://desaixie.github.io/pa-vdm/.
67089c42d08850e483b55c12
Desai Xie
claimed_verified
2024-10-11T07:54:17.886Z
6279f597812ee439d9c8de67
/avatars/49bea87294118e18df7b774f716af81a.svg
Desai Xie
user
desaix
67089c42d08850e483b55c13
Zhan Xu
67089c42d08850e483b55c14
Yicong Hong
admin_assigned
2024-10-11T10:55:47.230Z
654ca66fe1671abcbc50a6e4
https://cdn-avatars.huggingface.co/v1/production/uploads/654ca66fe1671abcbc50a6e4/VqcnC17YcqC9Hk65_LxtE.jpeg
Yicong Hong
user
YicongHong
67089c42d08850e483b55c15
Hao Tan
67089c42d08850e483b55c16
Difan Liu
admin_assigned
2024-10-11T10:55:41.875Z
6393a754d31aec207924611b
/avatars/51afe808f8e8a032e34ae6d7a2d6e407.svg
Difan Liu
user
smebliu
67089c42d08850e483b55c17
Feng Liu
67089c42d08850e483b55c18
Arie Kaufman
67089c42d08850e483b55c19
Yang Zhou
Progressive Autoregressive Video Diffusion Models
Current frontier video diffusion models have demonstrated remarkable results
at generating high-quality videos. However, they can only generate short video
clips, normally around 10 seconds or 240 frames, due to computation limitations
during training. In this work, we show that existing models can be naturally
extended to autoregressive video diffusion models without changing the
architectures. Our key idea is to assign the latent frames with progressively
increasing noise levels rather than a single noise level, which allows for
fine-grained condition among the latents and large overlaps between the
attention windows. Such progressive video denoising allows our models to
autoregressively generate video frames without quality degradation or abrupt
scene changes. We present state-of-the-art results on long video generation at
1 minute (1440 frames at 24 FPS). Videos from this paper are available at
https://desaixie.github.io/pa-vdm/.
Progressive noise assignment in autoregressive video diffusion models enables high-quality generation of long videos without quality degradation or abrupt scene changes.
video diffusion models autoregressive video diffusion models latent frames noise levels progressive noise assignment fine-grained condition attention windows video denoising long video generation