title: Background Prompting for Improved Object Depth
summary: Estimating the depth of objects from a single image is a valuable task for
  many vision, robotics, and graphics applications. However, current methods
  often fail to produce accurate depth for objects in diverse scenes. In this
  work, we propose a simple yet effective Background Prompting strategy that
  adapts the input object image with a learned background. We learn the
  background prompts only using small-scale synthetic object datasets. To infer
  object depth on a real image, we place the segmented object into the learned
  background prompt and run off-the-shelf depth networks. Background Prompting
  helps the depth networks focus on the foreground object, as they are made
  invariant to background variations. Moreover, Background Prompting minimizes
  the domain gap between synthetic and real object images, leading to better
  sim2real generalization than simple finetuning. Results on multiple synthetic
  and real datasets demonstrate consistent improvements in real object depths for
  a variety of existing depth networks. Code and optimized background prompts can
  be found at: https://mbaradad.github.io/depth_prompt.
paper_authors._id: 6482c405ba6c556892041f84
paper_authors.name: Manel Baradad
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-09T15:31:48.005Z
paper_authors._id: 6261a12c18176506fab39eba
paper_authors.avatarUrl: /avatars/e2b33b1fe6f43919dcb4df3410deda71.svg
paper_authors.fullname: Manel Baradad
paper_authors.type: user
paper_authors.user: manebaradad
paper_authors._id: 6482c405ba6c556892041f85
paper_authors.name: Yuanzhen Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-12T13:02:44.252Z
paper_authors._id: 6334ee5831a2be3938c85639
paper_authors.avatarUrl: /avatars/9821af26a585f54b595ec39f061f2250.svg
paper_authors.fullname: Yuanzhen Li
paper_authors.type: user
paper_authors.user: yzli
paper_authors._id: 6482c405ba6c556892041f86
paper_authors.name: Forrester Cole
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-12T13:03:04.407Z
paper_authors._id: 635003011d81beb8e244e68d
paper_authors.avatarUrl: /avatars/32f46e6778d1c52ebdd6f2511ba48257.svg
paper_authors.fullname: Forrester Cole
paper_authors.type: user
paper_authors.user: fcole
paper_authors._id: 6482c405ba6c556892041f87
paper_authors.name: Michael Rubinstein
paper_authors._id: 6482c405ba6c556892041f88
paper_authors.name: Antonio Torralba
paper_authors._id: 6482c405ba6c556892041f89
paper_authors.name: William T. Freeman
paper_authors._id: 6482c405ba6c556892041f8a
paper_authors.name: Varun Jampani
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-12T13:04:10.813Z
paper_authors._id: 630e7cfe3fc17ffc50f6e835
paper_authors.avatarUrl: /avatars/0b742ff094a09f9374fafcd97ab9e002.svg
paper_authors.fullname: Varun Jampani
paper_authors.type: user
paper_authors.user: varunjampani
paper_title: Background Prompting for Improved Object Depth
paper_summary: Estimating the depth of objects from a single image is a valuable task for
  many vision, robotics, and graphics applications. However, current methods
  often fail to produce accurate depth for objects in diverse scenes. In this
  work, we propose a simple yet effective Background Prompting strategy that
  adapts the input object image with a learned background. We learn the
  background prompts only using small-scale synthetic object datasets. To infer
  object depth on a real image, we place the segmented object into the learned
  background prompt and run off-the-shelf depth networks. Background Prompting
  helps the depth networks focus on the foreground object, as they are made
  invariant to background variations. Moreover, Background Prompting minimizes
  the domain gap between synthetic and real object images, leading to better
  sim2real generalization than simple finetuning. Results on multiple synthetic
  and real datasets demonstrate consistent improvements in real object depths for
  a variety of existing depth networks. Code and optimized background prompts can
  be found at: https://mbaradad.github.io/depth_prompt.
paper_ai_keywords: Background Prompting depth networks domain gap sim2real generalization