Depth Pro: Sharp Monocular Metric Depth in Less Than a Second
<|reference_start|>Depth Pro: Sharp Monocular Metric Depth in Less Than a Second: We present a foundation model for zero-shot metric monocular depth estimation. Our model, Depth Pro, synthesizes high-resolution depth maps with unparalleled sharpness and high-frequency details. The predictions are metric, with absolute scale, without relying on the availability of metadata such as camera intrinsics. And the model is fast, producing a 2.25-megapixel depth map in 0.3 seconds on a standard GPU. These characteristics are enabled by a number of technical contributions, including an efficient multi-scale vision transformer for dense prediction, a training protocol that combines real and synthetic datasets to achieve high metric accuracy alongside fine boundary tracing, dedicated evaluation metrics for boundary accuracy in estimated depth maps, and state-of-the-art focal length estimation from a single image. Extensive experiments analyze specific design choices and demonstrate that Depth Pro outperforms prior work along multiple dimensions. We release code and weights at https://github.com/apple/ml-depth-pro<|reference_end|>
@article{bochkovskii2024depth,
    title={Depth Pro: Sharp Monocular Metric Depth in Less Than a Second},
    author={Aleksei Bochkovskii and Ama"el Delaunoy and Hugo Germain and Marcel
  Santos and Yichao Zhou and Stephan R. Richter and Vladlen Koltun},
    journal={arXiv preprint arXiv:2410.02073},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.02073},
    primaryClass={cs.CV cs.LG}
}