title: Video World Models with Long-term Spatial Memory
summary: Emerging world models autoregressively generate video frames in response to
  actions, such as camera movements and text prompts, among other control
  signals. Due to limited temporal context window sizes, these models often
  struggle to maintain scene consistency during revisits, leading to severe
  forgetting of previously generated environments. Inspired by the mechanisms of
  human memory, we introduce a novel framework to enhancing long-term consistency
  of video world models through a geometry-grounded long-term spatial memory. Our
  framework includes mechanisms to store and retrieve information from the
  long-term spatial memory and we curate custom datasets to train and evaluate
  world models with explicitly stored 3D memory mechanisms. Our evaluations show
  improved quality, consistency, and context length compared to relevant
  baselines, paving the way towards long-term consistent world generation.
paper_authors._id: 6842929c46106f29d78635ad
paper_authors.name: Tong Wu
paper_authors._id: 6842929c46106f29d78635ae
paper_authors.name: Shuai Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-07T05:48:43.406Z
paper_authors._id: 6373037cd3ba6bd3f9bc32fa
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6373037cd3ba6bd3f9bc32fa/dIL4AIcL24QL_Rp4B9ewS.png
paper_authors.fullname: Shuai Yang
paper_authors.type: user
paper_authors.user: ysmikey
paper_authors._id: 6842929c46106f29d78635af
paper_authors.name: Ryan Po
paper_authors._id: 6842929c46106f29d78635b0
paper_authors.name: Yinghao Xu
paper_authors._id: 6842929c46106f29d78635b1
paper_authors.name: Ziwei Liu
paper_authors._id: 6842929c46106f29d78635b2
paper_authors.name: Dahua Lin
paper_authors._id: 6842929c46106f29d78635b3
paper_authors.name: Gordon Wetzstein
paper_title: Video World Models with Long-term Spatial Memory
paper_summary: Emerging world models autoregressively generate video frames in response to
  actions, such as camera movements and text prompts, among other control
  signals. Due to limited temporal context window sizes, these models often
  struggle to maintain scene consistency during revisits, leading to severe
  forgetting of previously generated environments. Inspired by the mechanisms of
  human memory, we introduce a novel framework to enhancing long-term consistency
  of video world models through a geometry-grounded long-term spatial memory. Our
  framework includes mechanisms to store and retrieve information from the
  long-term spatial memory and we curate custom datasets to train and evaluate
  world models with explicitly stored 3D memory mechanisms. Our evaluations show
  improved quality, consistency, and context length compared to relevant
  baselines, paving the way towards long-term consistent world generation.
paper_ai_summary: A new framework enhances video world models' long-term consistency by integrating a geometry-grounded long-term spatial memory mechanism.
paper_ai_keywords: world models autoregressive generation video frames control signals temporal context window scene consistency long-term spatial memory custom datasets 3D memory mechanisms