title: MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimation
summary: We release MiDaS v3.1 for monocular depth estimation, offering a variety of
  new models based on different encoder backbones. This release is motivated by
  the success of transformers in computer vision, with a large variety of
  pretrained vision transformers now available. We explore how using the most
  promising vision transformers as image encoders impacts depth estimation
  quality and runtime of the MiDaS architecture. Our investigation also includes
  recent convolutional approaches that achieve comparable quality to vision
  transformers in image classification tasks. While the previous release MiDaS
  v3.0 solely leverages the vanilla vision transformer ViT, MiDaS v3.1 offers
  additional models based on BEiT, Swin, SwinV2, Next-ViT and LeViT. These models
  offer different performance-runtime tradeoffs. The best model improves the
  depth estimation quality by 28% while efficient models enable downstream tasks
  requiring high frame rates. We also describe the general process for
  integrating new backbones. A video summarizing the work can be found at
  https://youtu.be/UjaeNNFf9sE and the code is available at
  https://github.com/isl-org/MiDaS.
paper_authors._id: 64c36115b63057f1fd37944b
paper_authors.name: Reiner Birkl
paper_authors._id: 64c36115b63057f1fd37944c
paper_authors.name: Diana Wofk
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-04T07:26:20.146Z
paper_authors._id: 66d740a519214d743cc30335
paper_authors.avatarUrl: /avatars/bc0839af30a183d5fae0a353d02733e2.svg
paper_authors.fullname: Diana Wofk
paper_authors.type: user
paper_authors.user: dwofk
paper_authors._id: 64c36115b63057f1fd37944d
paper_authors.name: Matthias Müller
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-28T16:16:54.250Z
paper_authors._id: 634ef08d23788038c5efd351
paper_authors.avatarUrl: /avatars/c319b7359a90dba86c674cc2369c879f.svg
paper_authors.fullname: Matthias Mueller
paper_authors.type: user
paper_authors.user: thias15
paper_title: MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimation
paper_summary: We release MiDaS v3.1 for monocular depth estimation, offering a variety of
  new models based on different encoder backbones. This release is motivated by
  the success of transformers in computer vision, with a large variety of
  pretrained vision transformers now available. We explore how using the most
  promising vision transformers as image encoders impacts depth estimation
  quality and runtime of the MiDaS architecture. Our investigation also includes
  recent convolutional approaches that achieve comparable quality to vision
  transformers in image classification tasks. While the previous release MiDaS
  v3.0 solely leverages the vanilla vision transformer ViT, MiDaS v3.1 offers
  additional models based on BEiT, Swin, SwinV2, Next-ViT and LeViT. These models
  offer different performance-runtime tradeoffs. The best model improves the
  depth estimation quality by 28% while efficient models enable downstream tasks
  requiring high frame rates. We also describe the general process for
  integrating new backbones. A video summarizing the work can be found at
  https://youtu.be/UjaeNNFf9sE and the code is available at
  https://github.com/isl-org/MiDaS.
paper_ai_summary: MiDaS v3.1 improves monocular depth estimation using various vision transformers and convolutional approaches, providing performance-runtime tradeoffs.
paper_ai_keywords: monocular depth estimation transformers pretrained vision transformers ViT BEiT Swin SwinV2 Next-ViT LeViT