Error Analyses of Auto-Regressive Video Diffusion Models: A Unified
  Framework
A variety of Auto-Regressive Video Diffusion Models (ARVDM) have achieved
remarkable successes in generating realistic long-form videos. However,
theoretical analyses of these models remain scant. In this work, we develop
theoretical underpinnings for these models and use our insights to improve the
performance of existing models. We first develop Meta-ARVDM, a unified
framework of ARVDMs that subsumes most existing methods. Using Meta-ARVDM, we
analyze the KL-divergence between the videos generated by Meta-ARVDM and the
true videos. Our analysis uncovers two important phenomena inherent to ARVDM --
error accumulation and memory bottleneck. By deriving an information-theoretic
impossibility result, we show that the memory bottleneck phenomenon cannot be
avoided. To mitigate the memory bottleneck, we design various network
structures to explicitly use more past frames. We also achieve a significantly
improved trade-off between the mitigation of the memory bottleneck and the
inference efficiency by compressing the frames. Experimental results on DMLab
and Minecraft validate the efficacy of our methods. Our experiments also
demonstrate a Pareto-frontier between the error accumulation and memory
bottleneck across different methods.
67d7eba831dd5b46c3e6fdcb
Jing Wang
claimed_verified
2025-03-18T08:09:01.169Z
633c2310c0fb6fd232f0accf
https://cdn-avatars.huggingface.co/v1/production/uploads/633c2310c0fb6fd232f0accf/AnvEvxMtd-BpSZbaoA4ha.jpeg
Wang Jing
user
k-nick
67d7eba831dd5b46c3e6fdcc
Fengzhuo Zhang
admin_assigned
2025-03-18T09:34:37.712Z
64b8c1a995bd42c7707f7918
/avatars/08c2929f8f150ecd6f8e5a06c4cb9034.svg
Fengzhuo Zhang
user
Fengzhuo
67d7eba831dd5b46c3e6fdcd
Xiaoli Li
admin_assigned
2025-03-18T09:34:47.006Z
67aa01782183876b1ec5760f
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/hZd1iBn_2yjHVoavcXPQo.png
xiaolili
user
xiaolili
67d7eba831dd5b46c3e6fdce
Vincent Y. F. Tan
67d7eba831dd5b46c3e6fdcf
Tianyu Pang
admin_assigned
2025-03-18T09:35:17.732Z
661a4a556fb488fa078c60aa
/avatars/c77401fa9c6d2db896b4a337bb3f8add.svg
Tianyu Pang
user
TIanyupang
67d7eba831dd5b46c3e6fdd0
Chao Du
admin_assigned
2025-03-18T09:35:29.239Z
632407c892e07e3ca20aca28
/avatars/23b51b37b12b51a0947f687d1de4d3b5.svg
Chao Du
user
duchao
67d7eba831dd5b46c3e6fdd1
Aixin Sun
admin_assigned
2025-03-18T09:35:35.567Z
664aab898fa42b4fe70ebf52
/avatars/a38455fd17bbc74ce3111f2c3da9aa59.svg
Aixin Sun
user
aixinsun
67d7eba831dd5b46c3e6fdd2
Zhuoran Yang
extracted_pending
2025-03-17T09:30:18.764Z
6397873ec0b27f432db8693f
/avatars/1db65fe55002ad5c137c4a59bbcd239d.svg
Zhuoran Yang
user
zhuoran
Error Analyses of Auto-Regressive Video Diffusion Models: A Unified
  Framework
A variety of Auto-Regressive Video Diffusion Models (ARVDM) have achieved
remarkable successes in generating realistic long-form videos. However,
theoretical analyses of these models remain scant. In this work, we develop
theoretical underpinnings for these models and use our insights to improve the
performance of existing models. We first develop Meta-ARVDM, a unified
framework of ARVDMs that subsumes most existing methods. Using Meta-ARVDM, we
analyze the KL-divergence between the videos generated by Meta-ARVDM and the
true videos. Our analysis uncovers two important phenomena inherent to ARVDM --
error accumulation and memory bottleneck. By deriving an information-theoretic
impossibility result, we show that the memory bottleneck phenomenon cannot be
avoided. To mitigate the memory bottleneck, we design various network
structures to explicitly use more past frames. We also achieve a significantly
improved trade-off between the mitigation of the memory bottleneck and the
inference efficiency by compressing the frames. Experimental results on DMLab
and Minecraft validate the efficacy of our methods. Our experiments also
demonstrate a Pareto-frontier between the error accumulation and memory
bottleneck across different methods.
Theoretical analysis of Auto-Regressive Video Diffusion Models uncovers error accumulation and memory bottlenecks, leading to improved performance through enhanced network structures and frame compression.
Auto-Regressive Video Diffusion Models ARVDM Meta-ARVDM KL-divergence error accumulation memory bottleneck information-theoretic impossibility result frame compression DMLab Minecraft Pareto-frontier