Long Context Tuning for Video Generation
Recent advances in video generation can produce realistic, minute-long
single-shot videos with scalable diffusion transformers. However, real-world
narrative videos require multi-shot scenes with visual and dynamic consistency
across shots. In this work, we introduce Long Context Tuning (LCT), a training
paradigm that expands the context window of pre-trained single-shot video
diffusion models to learn scene-level consistency directly from data. Our
method expands full attention mechanisms from individual shots to encompass all
shots within a scene, incorporating interleaved 3D position embedding and an
asynchronous noise strategy, enabling both joint and auto-regressive shot
generation without additional parameters. Models with bidirectional attention
after LCT can further be fine-tuned with context-causal attention, facilitating
auto-regressive generation with efficient KV-cache. Experiments demonstrate
single-shot models after LCT can produce coherent multi-shot scenes and exhibit
emerging capabilities, including compositional generation and interactive shot
extension, paving the way for more practical visual content creation. See
https://guoyww.github.io/projects/long-context-video/ for more details.
67d3adc5c14480a46038cecf
Yuwei Guo
claimed_verified
2025-03-16T21:17:39.505Z
6371f83d5ffa922d638ef486
https://cdn-avatars.huggingface.co/v1/production/uploads/6371f83d5ffa922d638ef486/UzWl3xhVvtol3Rz1mjQ9t.jpeg
Yuwei Guo
user
guoyww
67d3adc5c14480a46038ced0
Ceyuan Yang
67d3adc5c14480a46038ced1
Ziyan Yang
admin_assigned
2025-03-14T10:54:45.430Z
63c98de85fdc575773c52098
/avatars/b66675e781695daed7c00a7c802c91f5.svg
Ziyan Yang
user
ziyany
67d3adc5c14480a46038ced2
Zhibei Ma
admin_assigned
2025-03-14T10:54:24.552Z
645d5a44680734460f9e7742
/avatars/66687e2c413acce55436d967071f8786.svg
Zhibei Ma
user
Brightmzb
67d3adc5c14480a46038ced3
Zhijie Lin
admin_assigned
2025-03-14T10:53:03.195Z
64415957bd0c9726529802f6
/avatars/1132d1ee68fb58ec635d57c8175caacd.svg
Zhijie Lin
user
Ikuinen
67d3adc5c14480a46038ced4
Zhenheng Yang
admin_assigned
2025-03-14T10:52:40.696Z
6421183b69a2c2933882d652
/avatars/66813a8fa22915087cccd4dbfb945ca7.svg
Zhenheng Yang
user
zhenheny
67d3adc5c14480a46038ced5
Dahua Lin
admin_assigned
2025-03-14T10:52:10.238Z
636317ed80c1a705a6eff396
/avatars/3db090e101b916d9256d0d3e043db71d.svg
Dahua Lin
user
lindahua
67d3adc5c14480a46038ced6
Lu Jiang
Long Context Tuning for Video Generation
Recent advances in video generation can produce realistic, minute-long
single-shot videos with scalable diffusion transformers. However, real-world
narrative videos require multi-shot scenes with visual and dynamic consistency
across shots. In this work, we introduce Long Context Tuning (LCT), a training
paradigm that expands the context window of pre-trained single-shot video
diffusion models to learn scene-level consistency directly from data. Our
method expands full attention mechanisms from individual shots to encompass all
shots within a scene, incorporating interleaved 3D position embedding and an
asynchronous noise strategy, enabling both joint and auto-regressive shot
generation without additional parameters. Models with bidirectional attention
after LCT can further be fine-tuned with context-causal attention, facilitating
auto-regressive generation with efficient KV-cache. Experiments demonstrate
single-shot models after LCT can produce coherent multi-shot scenes and exhibit
emerging capabilities, including compositional generation and interactive shot
extension, paving the way for more practical visual content creation. See
https://guoyww.github.io/projects/long-context-video/ for more details.
Long Context Tuning expands pre-trained single-shot video diffusion models to generate coherent multi-shot videos with full attention mechanisms and position embeddings, enabling joint and auto-regressive shot generation and compositional capabilities.
video diffusion models Long Context Tuning context window full attention mechanisms 3D position embedding asynchronous noise strategy bidirectional attention context-causal attention KV-cache compositional generation interactive shot extension