title: Prompting Depth Anything for 4K Resolution Accurate Metric Depth
    Estimation
summary: Prompts play a critical role in unleashing the power of language and vision
  foundation models for specific tasks. For the first time, we introduce
  prompting into depth foundation models, creating a new paradigm for metric
  depth estimation termed Prompt Depth Anything. Specifically, we use a low-cost
  LiDAR as the prompt to guide the Depth Anything model for accurate metric depth
  output, achieving up to 4K resolution. Our approach centers on a concise prompt
  fusion design that integrates the LiDAR at multiple scales within the depth
  decoder. To address training challenges posed by limited datasets containing
  both LiDAR depth and precise GT depth, we propose a scalable data pipeline that
  includes synthetic data LiDAR simulation and real data pseudo GT depth
  generation. Our approach sets new state-of-the-arts on the ARKitScenes and
  ScanNet++ datasets and benefits downstream applications, including 3D
  reconstruction and generalized robotic grasping.
paper_authors._id: 67638f91d5948d839a423200
paper_authors.name: Haotong Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-20T09:52:43.197Z
paper_authors._id: 6489a01b8de3f9d810b0154f
paper_authors.avatarUrl: /avatars/f7a0fc6816535945e11bac1212dd7b57.svg
paper_authors.fullname: Haotong Lin
paper_authors.type: user
paper_authors.user: haotongl
paper_authors._id: 67638f91d5948d839a423201
paper_authors.name: Sida Peng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-20T09:52:36.196Z
paper_authors._id: 62986ca2b58e71e2ac9b8f01
paper_authors.avatarUrl: /avatars/83944db5f3dbb6f47c47c46fb2cb2849.svg
paper_authors.fullname: Sida Peng
paper_authors.type: user
paper_authors.user: pengsida
paper_authors._id: 67638f91d5948d839a423202
paper_authors.name: Jingxiao Chen
paper_authors._id: 67638f91d5948d839a423203
paper_authors.name: Songyou Peng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-20T09:52:22.648Z
paper_authors._id: 620cb0722d8bc91e1cf2e6e7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1644998719617-noauth.jpeg
paper_authors.fullname: Songyou Peng
paper_authors.type: user
paper_authors.user: syp
paper_authors._id: 67638f91d5948d839a423204
paper_authors.name: Jiaming Sun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-20T09:52:15.584Z
paper_authors._id: 63141d8df839c69a68e73201
paper_authors.avatarUrl: /avatars/b25fca555d22987241fd40bb56b8247a.svg
paper_authors.fullname: Jiaming  Sun
paper_authors.type: user
paper_authors.user: jiamingsun
paper_authors._id: 67638f91d5948d839a423205
paper_authors.name: Minghuan Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-20T09:52:02.965Z
paper_authors._id: 630c8f0c8e3ff0c72327140a
paper_authors.avatarUrl: /avatars/998de36d102b8bec85624f209272f7f2.svg
paper_authors.fullname: Minghuan Liu
paper_authors.type: user
paper_authors.user: ericonaldo
paper_authors._id: 67638f91d5948d839a423206
paper_authors.name: Hujun Bao
paper_authors._id: 67638f91d5948d839a423207
paper_authors.name: Jiashi Feng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-20T09:51:49.781Z
paper_authors._id: 67298e44017b96a1d0101dc4
paper_authors.avatarUrl: /avatars/1f8ed1a3e911e6a3021087b9371d284c.svg
paper_authors.fullname: Jiashi Feng
paper_authors.type: user
paper_authors.user: jshfeng
paper_authors._id: 67638f91d5948d839a423208
paper_authors.name: Xiaowei Zhou
paper_authors._id: 67638f91d5948d839a423209
paper_authors.name: Bingyi Kang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-20T08:33:34.930Z
paper_authors._id: 647b5fef6a79fbf5e996c47c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/647b5fef6a79fbf5e996c47c/IkSMnDsCY_CyEFCiMDuxe.jpeg
paper_authors.fullname: Bingyi Kang
paper_authors.type: user
paper_authors.user: bykang
paper_title: Prompting Depth Anything for 4K Resolution Accurate Metric Depth
    Estimation
paper_summary: Prompts play a critical role in unleashing the power of language and vision
  foundation models for specific tasks. For the first time, we introduce
  prompting into depth foundation models, creating a new paradigm for metric
  depth estimation termed Prompt Depth Anything. Specifically, we use a low-cost
  LiDAR as the prompt to guide the Depth Anything model for accurate metric depth
  output, achieving up to 4K resolution. Our approach centers on a concise prompt
  fusion design that integrates the LiDAR at multiple scales within the depth
  decoder. To address training challenges posed by limited datasets containing
  both LiDAR depth and precise GT depth, we propose a scalable data pipeline that
  includes synthetic data LiDAR simulation and real data pseudo GT depth
  generation. Our approach sets new state-of-the-arts on the ARKitScenes and
  ScanNet++ datasets and benefits downstream applications, including 3D
  reconstruction and generalized robotic grasping.
paper_ai_summary: Prompt Depth Anything integrates LiDAR data into depth foundation models for high-resolution depth estimation, achieving state-of-the-art results on ARKitScenes and ScanNet++ datasets and improving downstream applications.
paper_ai_keywords: prompting depth foundation models Prompt Depth Anything LiDAR depth decoder synthetic data LiDAR simulation pseudo GT depth generation ARKitScenes ScanNet++ 3D reconstruction generalized robotic grasping