title: Zero-Shot Metric Depth with a Field-of-View Conditioned Diffusion Model
summary: While methods for monocular depth estimation have made significant strides on
  standard benchmarks, zero-shot metric depth estimation remains unsolved.
  Challenges include the joint modeling of indoor and outdoor scenes, which often
  exhibit significantly different distributions of RGB and depth, and the
  depth-scale ambiguity due to unknown camera intrinsics. Recent work has
  proposed specialized multi-head architectures for jointly modeling indoor and
  outdoor scenes. In contrast, we advocate a generic, task-agnostic diffusion
  model, with several advancements such as log-scale depth parameterization to
  enable joint modeling of indoor and outdoor scenes, conditioning on the
  field-of-view (FOV) to handle scale ambiguity and synthetically augmenting FOV
  during training to generalize beyond the limited camera intrinsics in training
  datasets. Furthermore, by employing a more diverse training mixture than is
  common, and an efficient diffusion parameterization, our method, DMD (Diffusion
  for Metric Depth) achieves a 25\% reduction in relative error (REL) on
  zero-shot indoor and 33\% reduction on zero-shot outdoor datasets over the
  current SOTA using only a small number of denoising steps. For an overview see
  https://diffusion-vision.github.io/dmd
paper_authors._id: 6583c64883a9e1460c6ba00b
paper_authors.name: Saurabh Saxena
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-21T09:29:41.945Z
paper_authors._id: 6480af48de559d48dbb20d95
paper_authors.avatarUrl: /avatars/69f1e19921ba3ecdcd40bf55a6196aca.svg
paper_authors.fullname: Saurabh Saxena
paper_authors.type: user
paper_authors.user: saurabhsaxena
paper_authors._id: 6583c64883a9e1460c6ba00c
paper_authors.name: Junhwa Hur
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-21T09:29:49.440Z
paper_authors._id: 647f3046b0e96764589e107a
paper_authors.avatarUrl: /avatars/1074ee5ec1b2d5d297406f9d455213c4.svg
paper_authors.fullname: Junhwa Hur
paper_authors.type: user
paper_authors.user: hurjunhwa
paper_authors._id: 6583c64883a9e1460c6ba00d
paper_authors.name: Charles Herrmann
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-21T09:29:58.473Z
paper_authors._id: 64c7f088c864d962eddeaf8c
paper_authors.avatarUrl: /avatars/e6364c56e6cb6907b6dd2d34eed83720.svg
paper_authors.fullname: Charles Herrmann
paper_authors.type: user
paper_authors.user: irwinherrmann
paper_authors._id: 6583c64883a9e1460c6ba00e
paper_authors.name: Deqing Sun
paper_authors._id: 6583c64883a9e1460c6ba00f
paper_authors.name: David J. Fleet
paper_title: Zero-Shot Metric Depth with a Field-of-View Conditioned Diffusion Model
paper_summary: While methods for monocular depth estimation have made significant strides on
  standard benchmarks, zero-shot metric depth estimation remains unsolved.
  Challenges include the joint modeling of indoor and outdoor scenes, which often
  exhibit significantly different distributions of RGB and depth, and the
  depth-scale ambiguity due to unknown camera intrinsics. Recent work has
  proposed specialized multi-head architectures for jointly modeling indoor and
  outdoor scenes. In contrast, we advocate a generic, task-agnostic diffusion
  model, with several advancements such as log-scale depth parameterization to
  enable joint modeling of indoor and outdoor scenes, conditioning on the
  field-of-view (FOV) to handle scale ambiguity and synthetically augmenting FOV
  during training to generalize beyond the limited camera intrinsics in training
  datasets. Furthermore, by employing a more diverse training mixture than is
  common, and an efficient diffusion parameterization, our method, DMD (Diffusion
  for Metric Depth) achieves a 25\% reduction in relative error (REL) on
  zero-shot indoor and 33\% reduction on zero-shot outdoor datasets over the
  current SOTA using only a small number of denoising steps. For an overview see
  https://diffusion-vision.github.io/dmd
paper_ai_keywords: diffusion model monocular depth estimation zero-shot metric depth estimation indoor and outdoor scenes depth-scale ambiguity camera intrinsics multi-head architecture log-scale depth parameterization field-of-view (FOV) synthetic augmentation diffusion parameterization DMD (Diffusion for Metric Depth)