title: GeometryCrafter: Consistent Geometry Estimation for Open-world Videos
    with Diffusion Priors
summary: Despite remarkable advancements in video depth estimation, existing methods
  exhibit inherent limitations in achieving geometric fidelity through the
  affine-invariant predictions, limiting their applicability in reconstruction
  and other metrically grounded downstream tasks. We propose GeometryCrafter, a
  novel framework that recovers high-fidelity point map sequences with temporal
  coherence from open-world videos, enabling accurate 3D/4D reconstruction,
  camera parameter estimation, and other depth-based applications. At the core of
  our approach lies a point map Variational Autoencoder (VAE) that learns a
  latent space agnostic to video latent distributions for effective point map
  encoding and decoding. Leveraging the VAE, we train a video diffusion model to
  model the distribution of point map sequences conditioned on the input videos.
  Extensive evaluations on diverse datasets demonstrate that GeometryCrafter
  achieves state-of-the-art 3D accuracy, temporal consistency, and generalization
  capability.
paper_authors._id: 67ec958ebb1d6dd924f94a31
paper_authors.name: Tian-Xing Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T09:51:06.226Z
paper_authors._id: 65f8e4778dc7bb5b4db97f92
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/gBmQdovmANmjV72k3gaW8.png
paper_authors.fullname: Tian-Xing Xu
paper_authors.type: user
paper_authors.user: slothfulxtx
paper_authors._id: 67ec958ebb1d6dd924f94a32
paper_authors.name: Xiangjun Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T09:51:14.978Z
paper_authors._id: 64c0953a8137192a1e2474dc
paper_authors.avatarUrl: /avatars/546405a7eaf2f60ad108ceaa0dda7d08.svg
paper_authors.fullname: xiangjun gao
paper_authors.type: user
paper_authors.user: xiangjun0211
paper_authors._id: 67ec958ebb1d6dd924f94a33
paper_authors.name: Wenbo Hu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-02T08:22:52.313Z
paper_authors._id: 657a7458afbb0117ba15c59f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/657a7458afbb0117ba15c59f/8_iwTS1UG_mKnfylFbLsY.jpeg
paper_authors.fullname: Wenbo Hu
paper_authors.type: user
paper_authors.user: wbhu-tc
paper_authors._id: 67ec958ebb1d6dd924f94a34
paper_authors.name: Xiaoyu Li
paper_authors._id: 67ec958ebb1d6dd924f94a35
paper_authors.name: Song-Hai Zhang
paper_authors._id: 67ec958ebb1d6dd924f94a36
paper_authors.name: Ying Shan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T09:51:45.869Z
paper_authors._id: 63ca3ddc04c979828310bfcb
paper_authors.avatarUrl: /avatars/615e0d8622950b4408b40d550f02a894.svg
paper_authors.fullname: Ying Shan
paper_authors.type: user
paper_authors.user: yshan2u
paper_title: GeometryCrafter: Consistent Geometry Estimation for Open-world Videos
    with Diffusion Priors
paper_summary: Despite remarkable advancements in video depth estimation, existing methods
  exhibit inherent limitations in achieving geometric fidelity through the
  affine-invariant predictions, limiting their applicability in reconstruction
  and other metrically grounded downstream tasks. We propose GeometryCrafter, a
  novel framework that recovers high-fidelity point map sequences with temporal
  coherence from open-world videos, enabling accurate 3D/4D reconstruction,
  camera parameter estimation, and other depth-based applications. At the core of
  our approach lies a point map Variational Autoencoder (VAE) that learns a
  latent space agnostic to video latent distributions for effective point map
  encoding and decoding. Leveraging the VAE, we train a video diffusion model to
  model the distribution of point map sequences conditioned on the input videos.
  Extensive evaluations on diverse datasets demonstrate that GeometryCrafter
  achieves state-of-the-art 3D accuracy, temporal consistency, and generalization
  capability.
paper_ai_summary: GeometryCrafter uses a point map VAE and video diffusion model to estimate high-fidelity, temporally coherent depth maps from open-world videos, enhancing 3D reconstruction and camera parameter estimation.
paper_ai_keywords: point map VAE video diffusion model high-fidelity point maps temporal coherence 3D/4D reconstruction camera parameter estimation latent space distribution modeling