title: UniGeo: Taming Video Diffusion for Unified Consistent Geometry
    Estimation
summary: Recently, methods leveraging diffusion model priors to assist monocular
  geometric estimation (e.g., depth and normal) have gained significant attention
  due to their strong generalization ability. However, most existing works focus
  on estimating geometric properties within the camera coordinate system of
  individual video frames, neglecting the inherent ability of diffusion models to
  determine inter-frame correspondence. In this work, we demonstrate that,
  through appropriate design and fine-tuning, the intrinsic consistency of video
  generation models can be effectively harnessed for consistent geometric
  estimation. Specifically, we 1) select geometric attributes in the global
  coordinate system that share the same correspondence with video frames as the
  prediction targets, 2) introduce a novel and efficient conditioning method by
  reusing positional encodings, and 3) enhance performance through joint training
  on multiple geometric attributes that share the same correspondence. Our
  results achieve superior performance in predicting global geometric attributes
  in videos and can be directly applied to reconstruction tasks. Even when
  trained solely on static video data, our approach exhibits the potential to
  generalize to dynamic video scenes.
paper_authors._id: 683d11d1495f0b58f2fd49a9
paper_authors.name: Yang-Tian Sun
paper_authors._id: 683d11d1495f0b58f2fd49aa
paper_authors.name: Xin Yu
paper_authors._id: 683d11d1495f0b58f2fd49ab
paper_authors.name: Zehuan Huang
paper_authors._id: 683d11d1495f0b58f2fd49ac
paper_authors.name: Yi-Hua Huang
paper_authors._id: 683d11d1495f0b58f2fd49ad
paper_authors.name: Yuan-Chen Guo
paper_authors._id: 683d11d1495f0b58f2fd49ae
paper_authors.name: Ziyi Yang
paper_authors._id: 683d11d1495f0b58f2fd49af
paper_authors.name: Yan-Pei Cao
paper_authors._id: 683d11d1495f0b58f2fd49b0
paper_authors.name: Xiaojuan Qi
paper_title: UniGeo: Taming Video Diffusion for Unified Consistent Geometry
    Estimation
paper_summary: Recently, methods leveraging diffusion model priors to assist monocular
  geometric estimation (e.g., depth and normal) have gained significant attention
  due to their strong generalization ability. However, most existing works focus
  on estimating geometric properties within the camera coordinate system of
  individual video frames, neglecting the inherent ability of diffusion models to
  determine inter-frame correspondence. In this work, we demonstrate that,
  through appropriate design and fine-tuning, the intrinsic consistency of video
  generation models can be effectively harnessed for consistent geometric
  estimation. Specifically, we 1) select geometric attributes in the global
  coordinate system that share the same correspondence with video frames as the
  prediction targets, 2) introduce a novel and efficient conditioning method by
  reusing positional encodings, and 3) enhance performance through joint training
  on multiple geometric attributes that share the same correspondence. Our
  results achieve superior performance in predicting global geometric attributes
  in videos and can be directly applied to reconstruction tasks. Even when
  trained solely on static video data, our approach exhibits the potential to
  generalize to dynamic video scenes.
paper_ai_summary: Video generation models leveraging diffusion priors achieve superior global geometric attribute estimation and reconstructions, benefiting from inter-frame consistency and joint training on shared attributes.
paper_ai_keywords: diffusion models monocular geometric estimation depth normal camera coordinate system intrinsic consistency video generation models global coordinate system positional encodings joint training static video data dynamic video scenes