TokensGen: Harnessing Condensed Tokens for Long Video Generation
Generating consistent long videos is a complex challenge: while
diffusion-based generative models generate visually impressive short clips,
extending them to longer durations often leads to memory bottlenecks and
long-term inconsistency. In this paper, we propose TokensGen, a novel two-stage
framework that leverages condensed tokens to address these issues. Our method
decomposes long video generation into three core tasks: (1) inner-clip semantic
control, (2) long-term consistency control, and (3) inter-clip smooth
transition. First, we train To2V (Token-to-Video), a short video diffusion
model guided by text and video tokens, with a Video Tokenizer that condenses
short clips into semantically rich tokens. Second, we introduce T2To
(Text-to-Token), a video token diffusion transformer that generates all tokens
at once, ensuring global consistency across clips. Finally, during inference,
an adaptive FIFO-Diffusion strategy seamlessly connects adjacent clips,
reducing boundary artifacts and enhancing smooth transitions. Experimental
results demonstrate that our approach significantly enhances long-term temporal
and content coherence without incurring prohibitive computational overhead. By
leveraging condensed tokens and pre-trained short video models, our method
provides a scalable, modular solution for long video generation, opening new
possibilities for storytelling, cinematic production, and immersive
simulations. Please see our project page at
https://vicky0522.github.io/tokensgen-webpage/ .
687f8cd16b4cbf38ad40e068
Wenqi Ouyang
claimed_verified
2025-07-22T13:22:31.712Z
6487d9415d164d8c7f905bcf
/avatars/1c01dd4b88f88a9863a12c09a9a28bed.svg
Wenqi Ouyang
user
Vicky0522
687f8cd16b4cbf38ad40e069
Zeqi Xiao
687f8cd16b4cbf38ad40e06a
Danni Yang
687f8cd16b4cbf38ad40e06b
Yifan Zhou
687f8cd16b4cbf38ad40e06c
Shuai Yang
687f8cd16b4cbf38ad40e06d
Lei Yang
687f8cd16b4cbf38ad40e06e
Jianlou Si
687f8cd16b4cbf38ad40e06f
Xingang Pan
TokensGen: Harnessing Condensed Tokens for Long Video Generation
Generating consistent long videos is a complex challenge: while
diffusion-based generative models generate visually impressive short clips,
extending them to longer durations often leads to memory bottlenecks and
long-term inconsistency. In this paper, we propose TokensGen, a novel two-stage
framework that leverages condensed tokens to address these issues. Our method
decomposes long video generation into three core tasks: (1) inner-clip semantic
control, (2) long-term consistency control, and (3) inter-clip smooth
transition. First, we train To2V (Token-to-Video), a short video diffusion
model guided by text and video tokens, with a Video Tokenizer that condenses
short clips into semantically rich tokens. Second, we introduce T2To
(Text-to-Token), a video token diffusion transformer that generates all tokens
at once, ensuring global consistency across clips. Finally, during inference,
an adaptive FIFO-Diffusion strategy seamlessly connects adjacent clips,
reducing boundary artifacts and enhancing smooth transitions. Experimental
results demonstrate that our approach significantly enhances long-term temporal
and content coherence without incurring prohibitive computational overhead. By
leveraging condensed tokens and pre-trained short video models, our method
provides a scalable, modular solution for long video generation, opening new
possibilities for storytelling, cinematic production, and immersive
simulations. Please see our project page at
https://vicky0522.github.io/tokensgen-webpage/ .
TokensGen uses a two-stage framework with condensed tokens to generate long, consistent videos by addressing semantic control, long-term consistency, and smooth transitions.
diffusion-based generative models TokensGen To2V Token-to-Video Video Tokenizer T2To Text-to-Token video token diffusion transformer FIFO-Diffusion long-term temporal coherence content coherence