title: Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion
summary: Depth completion upgrades sparse depth measurements into dense depth maps
  guided by a conventional image. Existing methods for this highly ill-posed task
  operate in tightly constrained settings and tend to struggle when applied to
  images outside the training domain or when the available depth measurements are
  sparse, irregularly distributed, or of varying density. Inspired by recent
  advances in monocular depth estimation, we reframe depth completion as an
  image-conditional depth map generation guided by sparse measurements. Our
  method, Marigold-DC, builds on a pretrained latent diffusion model for
  monocular depth estimation and injects the depth observations as test-time
  guidance via an optimization scheme that runs in tandem with the iterative
  inference of denoising diffusion. The method exhibits excellent zero-shot
  generalization across a diverse range of environments and handles even
  extremely sparse guidance effectively. Our results suggest that contemporary
  monocular depth priors greatly robustify depth completion: it may be better to
  view the task as recovering dense depth from (dense) image pixels, guided by
  sparse depth; rather than as inpainting (sparse) depth, guided by an image.
  Project website: https://MarigoldDepthCompletion.github.io/
paper_authors._id: 6763737c606c173e18dc0675
paper_authors.name: Massimiliano Viola
paper_authors._id: 6763737c606c173e18dc0676
paper_authors.name: Kevin Qu
paper_authors._id: 6763737c606c173e18dc0677
paper_authors.name: Nando Metzger
paper_authors._id: 6763737c606c173e18dc0678
paper_authors.name: Bingxin Ke
paper_authors._id: 6763737c606c173e18dc0679
paper_authors.name: Alexander Becker
paper_authors._id: 6763737c606c173e18dc067a
paper_authors.name: Konrad Schindler
paper_authors._id: 6763737c606c173e18dc067b
paper_authors.name: Anton Obukhov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-19T18:01:48.399Z
paper_authors._id: 62f93abbc4817cfc0756b6f8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62f93abbc4817cfc0756b6f8/rGYLaq-rmoJJYotkC1VXk.jpeg
paper_authors.fullname: Anton Obukhov
paper_authors.type: user
paper_authors.user: toshas
paper_title: Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion
paper_summary: Depth completion upgrades sparse depth measurements into dense depth maps
  guided by a conventional image. Existing methods for this highly ill-posed task
  operate in tightly constrained settings and tend to struggle when applied to
  images outside the training domain or when the available depth measurements are
  sparse, irregularly distributed, or of varying density. Inspired by recent
  advances in monocular depth estimation, we reframe depth completion as an
  image-conditional depth map generation guided by sparse measurements. Our
  method, Marigold-DC, builds on a pretrained latent diffusion model for
  monocular depth estimation and injects the depth observations as test-time
  guidance via an optimization scheme that runs in tandem with the iterative
  inference of denoising diffusion. The method exhibits excellent zero-shot
  generalization across a diverse range of environments and handles even
  extremely sparse guidance effectively. Our results suggest that contemporary
  monocular depth priors greatly robustify depth completion: it may be better to
  view the task as recovering dense depth from (dense) image pixels, guided by
  sparse depth; rather than as inpainting (sparse) depth, guided by an image.
  Project website: https://MarigoldDepthCompletion.github.io/
paper_ai_keywords: latent diffusion model monocular depth estimation denoising diffusion