title: Video Depth Anything: Consistent Depth Estimation for Super-Long Videos
summary: Depth Anything has achieved remarkable success in monocular depth estimation
  with strong generalization ability. However, it suffers from temporal
  inconsistency in videos, hindering its practical applications. Various methods
  have been proposed to alleviate this issue by leveraging video generation
  models or introducing priors from optical flow and camera poses. Nonetheless,
  these methods are only applicable to short videos (< 10 seconds) and require a
  trade-off between quality and computational efficiency. We propose Video Depth
  Anything for high-quality, consistent depth estimation in super-long videos
  (over several minutes) without sacrificing efficiency. We base our model on
  Depth Anything V2 and replace its head with an efficient spatial-temporal head.
  We design a straightforward yet effective temporal consistency loss by
  constraining the temporal depth gradient, eliminating the need for additional
  geometric priors. The model is trained on a joint dataset of video depth and
  unlabeled images, similar to Depth Anything V2. Moreover, a novel
  key-frame-based strategy is developed for long video inference. Experiments
  show that our model can be applied to arbitrarily long videos without
  compromising quality, consistency, or generalization ability. Comprehensive
  evaluations on multiple video benchmarks demonstrate that our approach sets a
  new state-of-the-art in zero-shot video depth estimation. We offer models of
  different scales to support a range of scenarios, with our smallest model
  capable of real-time performance at 30 FPS.
paper_authors._id: 679069d37b150be8ddef0657
paper_authors.name: Sili Chen
paper_authors._id: 679069d37b150be8ddef0658
paper_authors.name: Hengkai Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T15:53:23.214Z
paper_authors._id: 65cc35785495933ab0edc096
paper_authors.avatarUrl: /avatars/2738e7f95e5929044fca74b218594d09.svg
paper_authors.fullname: Hengkai Guo
paper_authors.type: user
paper_authors.user: guohk10
paper_authors._id: 679069d37b150be8ddef0659
paper_authors.name: Shengnan Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T15:53:30.110Z
paper_authors._id: 678de89b94e4992bb9f2a303
paper_authors.avatarUrl: /avatars/68dc6a556d0ba59cc769320dc7c60a54.svg
paper_authors.fullname: Shengnan Zhu
paper_authors.type: user
paper_authors.user: Shane922
paper_authors._id: 679069d37b150be8ddef065a
paper_authors.name: Feihu Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T15:53:37.544Z
paper_authors._id: 674c3ac4e5965de680f691ed
paper_authors.avatarUrl: /avatars/04085e2c2de91f466eecec1d2c9f89e5.svg
paper_authors.fullname: feihuzhang
paper_authors.type: user
paper_authors.user: feihuzhang
paper_authors._id: 679069d37b150be8ddef065b
paper_authors.name: Zilong Huang
paper_authors._id: 679069d37b150be8ddef065c
paper_authors.name: Jiashi Feng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T15:54:06.632Z
paper_authors._id: 67298e44017b96a1d0101dc4
paper_authors.avatarUrl: /avatars/1f8ed1a3e911e6a3021087b9371d284c.svg
paper_authors.fullname: Jiashi Feng
paper_authors.type: user
paper_authors.user: jshfeng
paper_authors._id: 679069d37b150be8ddef065d
paper_authors.name: Bingyi Kang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T15:54:13.716Z
paper_authors._id: 647b5fef6a79fbf5e996c47c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/647b5fef6a79fbf5e996c47c/IkSMnDsCY_CyEFCiMDuxe.jpeg
paper_authors.fullname: Bingyi Kang
paper_authors.type: user
paper_authors.user: bykang
paper_title: Video Depth Anything: Consistent Depth Estimation for Super-Long Videos
paper_summary: Depth Anything has achieved remarkable success in monocular depth estimation
  with strong generalization ability. However, it suffers from temporal
  inconsistency in videos, hindering its practical applications. Various methods
  have been proposed to alleviate this issue by leveraging video generation
  models or introducing priors from optical flow and camera poses. Nonetheless,
  these methods are only applicable to short videos (< 10 seconds) and require a
  trade-off between quality and computational efficiency. We propose Video Depth
  Anything for high-quality, consistent depth estimation in super-long videos
  (over several minutes) without sacrificing efficiency. We base our model on
  Depth Anything V2 and replace its head with an efficient spatial-temporal head.
  We design a straightforward yet effective temporal consistency loss by
  constraining the temporal depth gradient, eliminating the need for additional
  geometric priors. The model is trained on a joint dataset of video depth and
  unlabeled images, similar to Depth Anything V2. Moreover, a novel
  key-frame-based strategy is developed for long video inference. Experiments
  show that our model can be applied to arbitrarily long videos without
  compromising quality, consistency, or generalization ability. Comprehensive
  evaluations on multiple video benchmarks demonstrate that our approach sets a
  new state-of-the-art in zero-shot video depth estimation. We offer models of
  different scales to support a range of scenarios, with our smallest model
  capable of real-time performance at 30 FPS.
paper_ai_keywords: monocular depth estimation temporal inconsistency video generation models optical flow camera poses Video Depth Anything Depth Anything V2 spatial-temporal head temporal consistency loss temporal depth gradient key-frame-based strategy zero-shot video depth estimation real-time performance