MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimation
We release MiDaS v3.1 for monocular depth estimation, offering a variety of
new models based on different encoder backbones. This release is motivated by
the success of transformers in computer vision, with a large variety of
pretrained vision transformers now available. We explore how using the most
promising vision transformers as image encoders impacts depth estimation
quality and runtime of the MiDaS architecture. Our investigation also includes
recent convolutional approaches that achieve comparable quality to vision
transformers in image classification tasks. While the previous release MiDaS
v3.0 solely leverages the vanilla vision transformer ViT, MiDaS v3.1 offers
additional models based on BEiT, Swin, SwinV2, Next-ViT and LeViT. These models
offer different performance-runtime tradeoffs. The best model improves the
depth estimation quality by 28% while efficient models enable downstream tasks
requiring high frame rates. We also describe the general process for
integrating new backbones. A video summarizing the work can be found at
https://youtu.be/UjaeNNFf9sE and the code is available at
https://github.com/isl-org/MiDaS.
64c36115b63057f1fd37944b
Reiner Birkl
64c36115b63057f1fd37944c
Diana Wofk
claimed_verified
2024-09-04T07:26:20.146Z
66d740a519214d743cc30335
/avatars/bc0839af30a183d5fae0a353d02733e2.svg
Diana Wofk
user
dwofk
64c36115b63057f1fd37944d
Matthias Müller
claimed_verified
2023-07-28T16:16:54.250Z
634ef08d23788038c5efd351
/avatars/c319b7359a90dba86c674cc2369c879f.svg
Matthias Mueller
user
thias15
MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimation
We release MiDaS v3.1 for monocular depth estimation, offering a variety of
new models based on different encoder backbones. This release is motivated by
the success of transformers in computer vision, with a large variety of
pretrained vision transformers now available. We explore how using the most
promising vision transformers as image encoders impacts depth estimation
quality and runtime of the MiDaS architecture. Our investigation also includes
recent convolutional approaches that achieve comparable quality to vision
transformers in image classification tasks. While the previous release MiDaS
v3.0 solely leverages the vanilla vision transformer ViT, MiDaS v3.1 offers
additional models based on BEiT, Swin, SwinV2, Next-ViT and LeViT. These models
offer different performance-runtime tradeoffs. The best model improves the
depth estimation quality by 28% while efficient models enable downstream tasks
requiring high frame rates. We also describe the general process for
integrating new backbones. A video summarizing the work can be found at
https://youtu.be/UjaeNNFf9sE and the code is available at
https://github.com/isl-org/MiDaS.
MiDaS v3.1 improves monocular depth estimation using various vision transformers and convolutional approaches, providing performance-runtime tradeoffs.
monocular depth estimation transformers pretrained vision transformers ViT BEiT Swin SwinV2 Next-ViT LeViT