title: Scaling Up Dynamic Human-Scene Interaction Modeling
summary: Confronting the challenges of data scarcity and advanced motion synthesis in
  human-scene interaction modeling, we introduce the TRUMANS dataset alongside a
  novel HSI motion synthesis method. TRUMANS stands as the most comprehensive
  motion-captured HSI dataset currently available, encompassing over 15 hours of
  human interactions across 100 indoor scenes. It intricately captures whole-body
  human motions and part-level object dynamics, focusing on the realism of
  contact. This dataset is further scaled up by transforming physical
  environments into exact virtual models and applying extensive augmentations to
  appearance and motion for both humans and objects while maintaining interaction
  fidelity. Utilizing TRUMANS, we devise a diffusion-based autoregressive model
  that efficiently generates HSI sequences of any length, taking into account
  both scene context and intended actions. In experiments, our approach shows
  remarkable zero-shot generalizability on a range of 3D scene datasets (e.g.,
  PROX, Replica, ScanNet, ScanNet++), producing motions that closely mimic
  original motion-captured sequences, as confirmed by quantitative experiments
  and human studies.
paper_authors._id: 65f27d5bdca94df6bdf9b89d
paper_authors.name: Nan Jiang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-18T08:35:33.995Z
paper_authors._id: 65f015dd6e38bf2ff9711f75
paper_authors.avatarUrl: /avatars/3a6afabfb94d78740ab93a6e3c803b7d.svg
paper_authors.fullname: Nan Jiang
paper_authors.type: user
paper_authors.user: jnnan
paper_authors._id: 65f27d5bdca94df6bdf9b89e
paper_authors.name: Zhiyuan Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-15T08:05:14.109Z
paper_authors._id: 648d628fba166fc8225a47c4
paper_authors.avatarUrl: /avatars/4d3c3cbc0d59035e6d96c60643a6f97f.svg
paper_authors.fullname: zhiyuan zhang
paper_authors.type: user
paper_authors.user: zzy-hugging
paper_authors._id: 65f27d5bdca94df6bdf9b89f
paper_authors.name: Hongjie Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:39:46.658Z
paper_authors._id: 6565d682995cc49553fa9f14
paper_authors.avatarUrl: /avatars/0b85c4f9316e2e8c9fea37a0df10089d.svg
paper_authors.fullname: Hongjie Li
paper_authors.type: user
paper_authors.user: awfuact
paper_authors._id: 65f27d5bdca94df6bdf9b8a0
paper_authors.name: Xiaoxuan Ma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:39:54.014Z
paper_authors._id: 6569840044ce94a7016f0abb
paper_authors.avatarUrl: /avatars/734a55c4077012701d5aff1510d440a2.svg
paper_authors.fullname: Xiaoxuan Ma
paper_authors.type: user
paper_authors.user: ShirleyMaxx
paper_authors._id: 65f27d5bdca94df6bdf9b8a1
paper_authors.name: Zan Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-14T12:31:51.219Z
paper_authors._id: 63a30e84528eba15b13f7355
paper_authors.avatarUrl: /avatars/79e26058c60fe0bc9f34f372b2e69ead.svg
paper_authors.fullname: Zan Wang
paper_authors.type: user
paper_authors.user: SuperZan
paper_authors._id: 65f27d5bdca94df6bdf9b8a2
paper_authors.name: Yixin Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:40:21.851Z
paper_authors._id: 64c396def6fe448b1ad553d6
paper_authors.avatarUrl: /avatars/b2ce4739f42dc00ee974fff7ee1cb301.svg
paper_authors.fullname: Yixin Chen
paper_authors.type: user
paper_authors.user: YixinChen
paper_authors._id: 65f27d5bdca94df6bdf9b8a3
paper_authors.name: Tengyu Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:40:30.501Z
paper_authors._id: 634379a81bdd3dfa55dcbe82
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1665366425318-noauth.png
paper_authors.fullname: Tengyu Liu
paper_authors.type: user
paper_authors.user: oxrider
paper_authors._id: 65f27d5bdca94df6bdf9b8a4
paper_authors.name: Yixin Zhu
paper_authors._id: 65f27d5bdca94df6bdf9b8a5
paper_authors.name: Siyuan Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:41:32.006Z
paper_authors._id: 63c7a33121bd95f80ed74652
paper_authors.avatarUrl: /avatars/7dd59afea785a2bff0ec2b757abd474e.svg
paper_authors.fullname: Siyuan Huang
paper_authors.type: user
paper_authors.user: thuhsy
paper_title: Scaling Up Dynamic Human-Scene Interaction Modeling
paper_summary: Confronting the challenges of data scarcity and advanced motion synthesis in
  human-scene interaction modeling, we introduce the TRUMANS dataset alongside a
  novel HSI motion synthesis method. TRUMANS stands as the most comprehensive
  motion-captured HSI dataset currently available, encompassing over 15 hours of
  human interactions across 100 indoor scenes. It intricately captures whole-body
  human motions and part-level object dynamics, focusing on the realism of
  contact. This dataset is further scaled up by transforming physical
  environments into exact virtual models and applying extensive augmentations to
  appearance and motion for both humans and objects while maintaining interaction
  fidelity. Utilizing TRUMANS, we devise a diffusion-based autoregressive model
  that efficiently generates HSI sequences of any length, taking into account
  both scene context and intended actions. In experiments, our approach shows
  remarkable zero-shot generalizability on a range of 3D scene datasets (e.g.,
  PROX, Replica, ScanNet, ScanNet++), producing motions that closely mimic
  original motion-captured sequences, as confirmed by quantitative experiments
  and human studies.
paper_ai_summary: A new diffusion-based autoregressive model using the TRUMANS dataset efficiently generates human-scene interaction motion sequences with high realism and zero-shot generalizability.
paper_ai_keywords: TRUMANS dataset diffusion-based autoregressive model human-scene interaction motion synthesis motion capture interaction fidelity scene context intended actions zero-shot generalizability PROX Replica ScanNet ScanNet++