Background Prompting for Improved Object Depth
Estimating the depth of objects from a single image is a valuable task for
many vision, robotics, and graphics applications. However, current methods
often fail to produce accurate depth for objects in diverse scenes. In this
work, we propose a simple yet effective Background Prompting strategy that
adapts the input object image with a learned background. We learn the
background prompts only using small-scale synthetic object datasets. To infer
object depth on a real image, we place the segmented object into the learned
background prompt and run off-the-shelf depth networks. Background Prompting
helps the depth networks focus on the foreground object, as they are made
invariant to background variations. Moreover, Background Prompting minimizes
the domain gap between synthetic and real object images, leading to better
sim2real generalization than simple finetuning. Results on multiple synthetic
and real datasets demonstrate consistent improvements in real object depths for
a variety of existing depth networks. Code and optimized background prompts can
be found at: https://mbaradad.github.io/depth_prompt.
6482c405ba6c556892041f84
Manel Baradad
claimed_verified
2023-06-09T15:31:48.005Z
6261a12c18176506fab39eba
/avatars/e2b33b1fe6f43919dcb4df3410deda71.svg
Manel Baradad
user
manebaradad
6482c405ba6c556892041f85
Yuanzhen Li
admin_assigned
2023-06-12T13:02:44.252Z
6334ee5831a2be3938c85639
/avatars/9821af26a585f54b595ec39f061f2250.svg
Yuanzhen Li
user
yzli
6482c405ba6c556892041f86
Forrester Cole
admin_assigned
2023-06-12T13:03:04.407Z
635003011d81beb8e244e68d
/avatars/32f46e6778d1c52ebdd6f2511ba48257.svg
Forrester Cole
user
fcole
6482c405ba6c556892041f87
Michael Rubinstein
6482c405ba6c556892041f88
Antonio Torralba
6482c405ba6c556892041f89
William T. Freeman
6482c405ba6c556892041f8a
Varun Jampani
admin_assigned
2023-06-12T13:04:10.813Z
630e7cfe3fc17ffc50f6e835
/avatars/0b742ff094a09f9374fafcd97ab9e002.svg
Varun Jampani
user
varunjampani
Background Prompting for Improved Object Depth
Estimating the depth of objects from a single image is a valuable task for
many vision, robotics, and graphics applications. However, current methods
often fail to produce accurate depth for objects in diverse scenes. In this
work, we propose a simple yet effective Background Prompting strategy that
adapts the input object image with a learned background. We learn the
background prompts only using small-scale synthetic object datasets. To infer
object depth on a real image, we place the segmented object into the learned
background prompt and run off-the-shelf depth networks. Background Prompting
helps the depth networks focus on the foreground object, as they are made
invariant to background variations. Moreover, Background Prompting minimizes
the domain gap between synthetic and real object images, leading to better
sim2real generalization than simple finetuning. Results on multiple synthetic
and real datasets demonstrate consistent improvements in real object depths for
a variety of existing depth networks. Code and optimized background prompts can
be found at: https://mbaradad.github.io/depth_prompt.
A novel Background Prompting strategy enhances depth estimation accuracy by adapting object images with learned backgrounds, improving focus and sim2real generalization.
Background Prompting depth networks segmented object domain gap sim2real generalization