title: DepthFM: Fast Monocular Depth Estimation with Flow Matching
summary: Monocular depth estimation is crucial for numerous downstream vision tasks
  and applications. Current discriminative approaches to this problem are limited
  due to blurry artifacts, while state-of-the-art generative methods suffer from
  slow sampling due to their SDE nature. Rather than starting from noise, we seek
  a direct mapping from input image to depth map. We observe that this can be
  effectively framed using flow matching, since its straight trajectories through
  solution space offer efficiency and high quality. Our study demonstrates that a
  pre-trained image diffusion model can serve as an adequate prior for a flow
  matching depth model, allowing efficient training on only synthetic data to
  generalize to real images. We find that an auxiliary surface normals loss
  further improves the depth estimates. Due to the generative nature of our
  approach, our model reliably predicts the confidence of its depth estimates. On
  standard benchmarks of complex natural scenes, our lightweight approach
  exhibits state-of-the-art performance at favorable low computational cost
  despite only being trained on little synthetic data.
paper_authors._id: 65fbbae8030389a29b7b9c68
paper_authors.name: Ming Gui
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-30T08:10:02.718Z
paper_authors._id: 6319f120d47392f426cd48be
paper_authors.avatarUrl: /avatars/fa6d8761ef7a1a86a8a7e97408dc3142.svg
paper_authors.fullname: Ming Gui
paper_authors.type: user
paper_authors.user: mgui
paper_authors._id: 65fbbae8030389a29b7b9c69
paper_authors.name: Johannes S. Fischer
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-22T09:30:16.872Z
paper_authors._id: 63cea5dd257cc48a04f8dc21
paper_authors.avatarUrl: /avatars/db697e0954ff5be392b8ab588df2f0c4.svg
paper_authors.fullname: J. Schusterbauer
paper_authors.type: user
paper_authors.user: joh-schb
paper_authors._id: 65fbbae8030389a29b7b9c6a
paper_authors.name: Ulrich Prestel
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T10:22:30.697Z
paper_authors._id: 6425e56785f26ab94af19797
paper_authors.avatarUrl: /avatars/555fafac9b555da7be1a112fac9a0cbf.svg
paper_authors.fullname: Ulrich Prestel
paper_authors.type: user
paper_authors.user: upr
paper_authors._id: 65fbbae8030389a29b7b9c6b
paper_authors.name: Pingchuan Ma
paper_authors._id: 65fbbae8030389a29b7b9c6c
paper_authors.name: Dmytro Kotovenko
paper_authors._id: 65fbbae8030389a29b7b9c6d
paper_authors.name: Olga Grebenkova
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-08T20:01:18.911Z
paper_authors._id: 633b481d0d68f86e2d96ed84
paper_authors.avatarUrl: /avatars/3c5beacabe55b7a7a9419e77324018dd.svg
paper_authors.fullname: Olga
paper_authors.type: user
paper_authors.user: magistrkoljan
paper_authors._id: 65fbbae8030389a29b7b9c6e
paper_authors.name: Stefan Andreas Baumann
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-25T09:44:14.282Z
paper_authors._id: 6471c12a0c2b5fdaf1f07c45
paper_authors.avatarUrl: /avatars/6783067212d24d1e716a8b4c64df61b4.svg
paper_authors.fullname: Stefan Baumann
paper_authors.type: user
paper_authors.user: stefan-baumann
paper_authors._id: 65fbbae8030389a29b7b9c6f
paper_authors.name: Vincent Tao Hu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T10:24:39.243Z
paper_authors._id: 63dcd7ac22cc06e76a8484ce
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63dcd7ac22cc06e76a8484ce/4O8NcevQ0ZWhJJ1r4Pbam.jpeg
paper_authors.fullname: Vincent Tao Hu
paper_authors.type: user
paper_authors.user: taohu
paper_authors._id: 65fbbae8030389a29b7b9c70
paper_authors.name: Björn Ommer
paper_title: DepthFM: Fast Monocular Depth Estimation with Flow Matching
paper_summary: Monocular depth estimation is crucial for numerous downstream vision tasks
  and applications. Current discriminative approaches to this problem are limited
  due to blurry artifacts, while state-of-the-art generative methods suffer from
  slow sampling due to their SDE nature. Rather than starting from noise, we seek
  a direct mapping from input image to depth map. We observe that this can be
  effectively framed using flow matching, since its straight trajectories through
  solution space offer efficiency and high quality. Our study demonstrates that a
  pre-trained image diffusion model can serve as an adequate prior for a flow
  matching depth model, allowing efficient training on only synthetic data to
  generalize to real images. We find that an auxiliary surface normals loss
  further improves the depth estimates. Due to the generative nature of our
  approach, our model reliably predicts the confidence of its depth estimates. On
  standard benchmarks of complex natural scenes, our lightweight approach
  exhibits state-of-the-art performance at favorable low computational cost
  despite only being trained on little synthetic data.
paper_ai_keywords: flow matching solution space image diffusion model flow matching depth model auxiliary surface normals loss