title: Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstruction
summary: We introduce Geo4D, a method to repurpose video diffusion models for
  monocular 3D reconstruction of dynamic scenes. By leveraging the strong dynamic
  prior captured by such video models, Geo4D can be trained using only synthetic
  data while generalizing well to real data in a zero-shot manner. Geo4D predicts
  several complementary geometric modalities, namely point, depth, and ray maps.
  It uses a new multi-modal alignment algorithm to align and fuse these
  modalities, as well as multiple sliding windows, at inference time, thus
  obtaining robust and accurate 4D reconstruction of long videos. Extensive
  experiments across multiple benchmarks show that Geo4D significantly surpasses
  state-of-the-art video depth estimation methods, including recent methods such
  as MonST3R, which are also designed to handle dynamic scenes.
paper_authors._id: 67f90d832af32ec4d45797cb
paper_authors.name: Zeren Jiang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-11T16:18:47.541Z
paper_authors._id: 679517e61703797636585760
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Dmd8ws39tRZLwUH7wwk8i.png
paper_authors.fullname: Zeren Jiang
paper_authors.type: user
paper_authors.user: jzr99
paper_authors._id: 67f90d832af32ec4d45797cc
paper_authors.name: Chuanxia Zheng
paper_authors._id: 67f90d832af32ec4d45797cd
paper_authors.name: Iro Laina
paper_authors._id: 67f90d832af32ec4d45797ce
paper_authors.name: Diane Larlus
paper_authors._id: 67f90d832af32ec4d45797cf
paper_authors.name: Andrea Vedaldi
paper_title: Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstruction
paper_summary: We introduce Geo4D, a method to repurpose video diffusion models for
  monocular 3D reconstruction of dynamic scenes. By leveraging the strong dynamic
  prior captured by such video models, Geo4D can be trained using only synthetic
  data while generalizing well to real data in a zero-shot manner. Geo4D predicts
  several complementary geometric modalities, namely point, depth, and ray maps.
  It uses a new multi-modal alignment algorithm to align and fuse these
  modalities, as well as multiple sliding windows, at inference time, thus
  obtaining robust and accurate 4D reconstruction of long videos. Extensive
  experiments across multiple benchmarks show that Geo4D significantly surpasses
  state-of-the-art video depth estimation methods, including recent methods such
  as MonST3R, which are also designed to handle dynamic scenes.
paper_ai_summary: Geo4D repurposes video diffusion models for zero-shot 3D reconstruction of dynamic scenes using synthetic data, achieving state-of-the-art performance in video depth estimation.
paper_ai_keywords: video diffusion models monocular 3D reconstruction dynamic scenes synthetic data zero-shot geometric modalities point maps depth maps ray maps multi-modal alignment sliding windows