Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion
Depth completion upgrades sparse depth measurements into dense depth maps
guided by a conventional image. Existing methods for this highly ill-posed task
operate in tightly constrained settings and tend to struggle when applied to
images outside the training domain or when the available depth measurements are
sparse, irregularly distributed, or of varying density. Inspired by recent
advances in monocular depth estimation, we reframe depth completion as an
image-conditional depth map generation guided by sparse measurements. Our
method, Marigold-DC, builds on a pretrained latent diffusion model for
monocular depth estimation and injects the depth observations as test-time
guidance via an optimization scheme that runs in tandem with the iterative
inference of denoising diffusion. The method exhibits excellent zero-shot
generalization across a diverse range of environments and handles even
extremely sparse guidance effectively. Our results suggest that contemporary
monocular depth priors greatly robustify depth completion: it may be better to
view the task as recovering dense depth from (dense) image pixels, guided by
sparse depth; rather than as inpainting (sparse) depth, guided by an image.
Project website: https://MarigoldDepthCompletion.github.io/
6763737c606c173e18dc0675
Massimiliano Viola
6763737c606c173e18dc0676
Kevin Qu
6763737c606c173e18dc0677
Nando Metzger
6763737c606c173e18dc0678
Bingxin Ke
6763737c606c173e18dc0679
Alexander Becker
6763737c606c173e18dc067a
Konrad Schindler
claimed_verified
2025-05-15T13:45:29.996Z
6750649b7edd6a98a1bbcd06
/avatars/ba27f12d0333cf2d400d4405af7efe97.svg
Konrad Schindler
user
konradschindler
6763737c606c173e18dc067b
Anton Obukhov
claimed_verified
2024-12-19T18:01:48.399Z
62f93abbc4817cfc0756b6f8
https://cdn-avatars.huggingface.co/v1/production/uploads/62f93abbc4817cfc0756b6f8/rGYLaq-rmoJJYotkC1VXk.jpeg
Anton Obukhov
user
toshas
Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion
Depth completion upgrades sparse depth measurements into dense depth maps
guided by a conventional image. Existing methods for this highly ill-posed task
operate in tightly constrained settings and tend to struggle when applied to
images outside the training domain or when the available depth measurements are
sparse, irregularly distributed, or of varying density. Inspired by recent
advances in monocular depth estimation, we reframe depth completion as an
image-conditional depth map generation guided by sparse measurements. Our
method, Marigold-DC, builds on a pretrained latent diffusion model for
monocular depth estimation and injects the depth observations as test-time
guidance via an optimization scheme that runs in tandem with the iterative
inference of denoising diffusion. The method exhibits excellent zero-shot
generalization across a diverse range of environments and handles even
extremely sparse guidance effectively. Our results suggest that contemporary
monocular depth priors greatly robustify depth completion: it may be better to
view the task as recovering dense depth from (dense) image pixels, guided by
sparse depth; rather than as inpainting (sparse) depth, guided by an image.
Project website: https://MarigoldDepthCompletion.github.io/
Marigold-DC uses a pretrained latent diffusion model to generate dense depth maps from sparse measurements, improving zero-shot generalization across diverse environments.
latent diffusion model monocular depth estimation depth map generation denoising diffusion zero-shot generalization