title: BetterDepth: Plug-and-Play Diffusion Refiner for Zero-Shot Monocular
    Depth Estimation
summary: By training over large-scale datasets, zero-shot monocular depth estimation
  (MDE) methods show robust performance in the wild but often suffer from
  insufficiently precise details. Although recent diffusion-based MDE approaches
  exhibit appealing detail extraction ability, they still struggle in
  geometrically challenging scenes due to the difficulty of gaining robust
  geometric priors from diverse datasets. To leverage the complementary merits of
  both worlds, we propose BetterDepth to efficiently achieve geometrically
  correct affine-invariant MDE performance while capturing fine-grained details.
  Specifically, BetterDepth is a conditional diffusion-based refiner that takes
  the prediction from pre-trained MDE models as depth conditioning, in which the
  global depth context is well-captured, and iteratively refines details based on
  the input image. For the training of such a refiner, we propose global
  pre-alignment and local patch masking methods to ensure the faithfulness of
  BetterDepth to depth conditioning while learning to capture fine-grained scene
  details. By efficient training on small-scale synthetic datasets, BetterDepth
  achieves state-of-the-art zero-shot MDE performance on diverse public datasets
  and in-the-wild scenes. Moreover, BetterDepth can improve the performance of
  other MDE models in a plug-and-play manner without additional re-training.
paper_authors._id: 66a355b0e88715bf1bb3094e
paper_authors.name: Xiang Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-26T10:26:32.542Z
paper_authors._id: 657dc1576dc01435cd9029d8
paper_authors.avatarUrl: /avatars/3bba11ac7659fce61aeaedf40e2057a8.svg
paper_authors.fullname: Xiang Zhang
paper_authors.type: user
paper_authors.user: XiangZ
paper_authors._id: 66a355b0e88715bf1bb3094f
paper_authors.name: Bingxin Ke
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-26T08:39:19.926Z
paper_authors._id: 63d90391da4f72339244c2a8
paper_authors.avatarUrl: /avatars/eb0e0259c391d59739c1a205c36bb539.svg
paper_authors.fullname: Bingxin Ke
paper_authors.type: user
paper_authors.user: Bingxin
paper_authors._id: 66a355b0e88715bf1bb30950
paper_authors.name: Hayko Riemenschneider
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-26T08:39:27.141Z
paper_authors._id: 6332ab213c1494be1137bd58
paper_authors.avatarUrl: /avatars/f65dac80d7f5148c46781055ddbdae51.svg
paper_authors.fullname: hayko riemenschneider
paper_authors.type: user
paper_authors.user: chrysmun
paper_authors._id: 66a355b0e88715bf1bb30951
paper_authors.name: Nando Metzger
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-26T08:39:32.523Z
paper_authors._id: 63a5785a8fb23d08bb2d0291
paper_authors.avatarUrl: /avatars/758b06dae06e9eee6fced10ce682aef1.svg
paper_authors.fullname: Nando Metzger
paper_authors.type: user
paper_authors.user: nandometzger
paper_authors._id: 66a355b0e88715bf1bb30952
paper_authors.name: Anton Obukhov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-26T08:05:55.086Z
paper_authors._id: 62f93abbc4817cfc0756b6f8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62f93abbc4817cfc0756b6f8/rGYLaq-rmoJJYotkC1VXk.jpeg
paper_authors.fullname: Anton Obukhov
paper_authors.type: user
paper_authors.user: toshas
paper_authors._id: 66a355b0e88715bf1bb30953
paper_authors.name: Markus Gross
paper_authors._id: 66a355b0e88715bf1bb30954
paper_authors.name: Konrad Schindler
paper_authors._id: 66a355b0e88715bf1bb30955
paper_authors.name: Christopher Schroers
paper_title: BetterDepth: Plug-and-Play Diffusion Refiner for Zero-Shot Monocular
    Depth Estimation
paper_summary: By training over large-scale datasets, zero-shot monocular depth estimation
  (MDE) methods show robust performance in the wild but often suffer from
  insufficiently precise details. Although recent diffusion-based MDE approaches
  exhibit appealing detail extraction ability, they still struggle in
  geometrically challenging scenes due to the difficulty of gaining robust
  geometric priors from diverse datasets. To leverage the complementary merits of
  both worlds, we propose BetterDepth to efficiently achieve geometrically
  correct affine-invariant MDE performance while capturing fine-grained details.
  Specifically, BetterDepth is a conditional diffusion-based refiner that takes
  the prediction from pre-trained MDE models as depth conditioning, in which the
  global depth context is well-captured, and iteratively refines details based on
  the input image. For the training of such a refiner, we propose global
  pre-alignment and local patch masking methods to ensure the faithfulness of
  BetterDepth to depth conditioning while learning to capture fine-grained scene
  details. By efficient training on small-scale synthetic datasets, BetterDepth
  achieves state-of-the-art zero-shot MDE performance on diverse public datasets
  and in-the-wild scenes. Moreover, BetterDepth can improve the performance of
  other MDE models in a plug-and-play manner without additional re-training.
paper_ai_keywords: zero-shot monocular depth estimation (MDE) diffusion-based MDE approaches affine-invariant MDE performance conditional diffusion-based refiner depth conditioning global depth context global pre-alignment local patch masking zero-shot MDE performance in-the-wild scenes plug-and-play manner