title: DECO: Dense Estimation of 3D Human-Scene Contact In The Wild
summary: Understanding how humans use physical contact to interact with the world is
  key to enabling human-centric artificial intelligence. While inferring 3D
  contact is crucial for modeling realistic and physically-plausible human-object
  interactions, existing methods either focus on 2D, consider body joints rather
  than the surface, use coarse 3D body regions, or do not generalize to
  in-the-wild images. In contrast, we focus on inferring dense, 3D contact
  between the full body surface and objects in arbitrary images. To achieve this,
  we first collect DAMON, a new dataset containing dense vertex-level contact
  annotations paired with RGB images containing complex human-object and
  human-scene contact. Second, we train DECO, a novel 3D contact detector that
  uses both body-part-driven and scene-context-driven attention to estimate
  vertex-level contact on the SMPL body. DECO builds on the insight that human
  observers recognize contact by reasoning about the contacting body parts, their
  proximity to scene objects, and the surrounding scene context. We perform
  extensive evaluations of our detector on DAMON as well as on the RICH and
  BEHAVE datasets. We significantly outperform existing SOTA methods across all
  benchmarks. We also show qualitatively that DECO generalizes well to diverse
  and challenging real-world human interactions in natural images. The code,
  data, and models are available at https://deco.is.tue.mpg.de.
paper_authors._id: 65151acef624f38acb2625c0
paper_authors.name: Shashank Tripathi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-29T07:03:48.529Z
paper_authors._id: 65156f9c71c52d030afed8ed
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/xRvgF_QigDV5POLrXdVZk.png
paper_authors.fullname: Shashank Tripathi
paper_authors.type: user
paper_authors.user: sha2nkt
paper_authors._id: 65151acef624f38acb2625c1
paper_authors.name: Agniv Chatterjee
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-28T09:17:05.749Z
paper_authors._id: 64d77df9505306fcd2da8d5b
paper_authors.avatarUrl: /avatars/9053d3330d916c08c4ebe3a0a590e64e.svg
paper_authors.fullname: Agniv Chatterjee
paper_authors.type: user
paper_authors.user: ac5113
paper_authors._id: 65151acef624f38acb2625c2
paper_authors.name: Jean-Claude Passy
paper_authors._id: 65151acef624f38acb2625c3
paper_authors.name: Hongwei Yi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-28T09:17:20.823Z
paper_authors._id: 6438d307d221ff12edb1e47e
paper_authors.avatarUrl: /avatars/638b20e9f7deec235aa2dcc56bc7ecb7.svg
paper_authors.fullname: Hongwei Yi
paper_authors.type: user
paper_authors.user: chairmenyi
paper_authors._id: 65151acef624f38acb2625c4
paper_authors.name: Dimitrios Tzionas
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-09-28T10:36:19.783Z
paper_authors._id: 6515533dee24d3b4e3a66f01
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/Jh-1CnLgDgUlkEYmcHaId.jpeg
paper_authors.fullname: Dimitrios Tzionas
paper_authors.type: user
paper_authors.user: dimtzionas
paper_authors._id: 65151acef624f38acb2625c5
paper_authors.name: Michael J. Black
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-28T09:17:33.417Z
paper_authors._id: 649fbababa7a05e3626546c8
paper_authors.avatarUrl: /avatars/a9aa4b4914649527094fc1b12779e014.svg
paper_authors.fullname: Michael J. Black
paper_authors.type: user
paper_authors.user: MichaelJBlack
paper_title: DECO: Dense Estimation of 3D Human-Scene Contact In The Wild
paper_summary: Understanding how humans use physical contact to interact with the world is
  key to enabling human-centric artificial intelligence. While inferring 3D
  contact is crucial for modeling realistic and physically-plausible human-object
  interactions, existing methods either focus on 2D, consider body joints rather
  than the surface, use coarse 3D body regions, or do not generalize to
  in-the-wild images. In contrast, we focus on inferring dense, 3D contact
  between the full body surface and objects in arbitrary images. To achieve this,
  we first collect DAMON, a new dataset containing dense vertex-level contact
  annotations paired with RGB images containing complex human-object and
  human-scene contact. Second, we train DECO, a novel 3D contact detector that
  uses both body-part-driven and scene-context-driven attention to estimate
  vertex-level contact on the SMPL body. DECO builds on the insight that human
  observers recognize contact by reasoning about the contacting body parts, their
  proximity to scene objects, and the surrounding scene context. We perform
  extensive evaluations of our detector on DAMON as well as on the RICH and
  BEHAVE datasets. We significantly outperform existing SOTA methods across all
  benchmarks. We also show qualitatively that DECO generalizes well to diverse
  and challenging real-world human interactions in natural images. The code,
  data, and models are available at https://deco.is.tue.mpg.de.
paper_ai_summary: A novel 3D contact detector, DECO, infers dense, vertex-level contact between the full body surface and objects in arbitrary images using body-part-driven and scene-context-driven attention, outperforming existing methods across benchmarks.
paper_ai_keywords: DAMON dataset vertex-level contact 3D contact detector body-part-driven attention scene-context-driven attention SMPL body DECO dense contact inferring 3D contact RICH dataset BEHAVE dataset