Scaling Up Dynamic Human-Scene Interaction Modeling
Confronting the challenges of data scarcity and advanced motion synthesis in
human-scene interaction modeling, we introduce the TRUMANS dataset alongside a
novel HSI motion synthesis method. TRUMANS stands as the most comprehensive
motion-captured HSI dataset currently available, encompassing over 15 hours of
human interactions across 100 indoor scenes. It intricately captures whole-body
human motions and part-level object dynamics, focusing on the realism of
contact. This dataset is further scaled up by transforming physical
environments into exact virtual models and applying extensive augmentations to
appearance and motion for both humans and objects while maintaining interaction
fidelity. Utilizing TRUMANS, we devise a diffusion-based autoregressive model
that efficiently generates HSI sequences of any length, taking into account
both scene context and intended actions. In experiments, our approach shows
remarkable zero-shot generalizability on a range of 3D scene datasets (e.g.,
PROX, Replica, ScanNet, ScanNet++), producing motions that closely mimic
original motion-captured sequences, as confirmed by quantitative experiments
and human studies.
65f27d5bdca94df6bdf9b89d
Nan Jiang
claimed_verified
2024-03-18T08:35:33.995Z
65f015dd6e38bf2ff9711f75
/avatars/3a6afabfb94d78740ab93a6e3c803b7d.svg
Nan Jiang
user
jnnan
65f27d5bdca94df6bdf9b89e
Zhiyuan Zhang
claimed_verified
2024-03-15T08:05:14.109Z
648d628fba166fc8225a47c4
/avatars/4d3c3cbc0d59035e6d96c60643a6f97f.svg
zhiyuan zhang
user
zzy-hugging
65f27d5bdca94df6bdf9b89f
Hongjie Li
admin_assigned
2024-03-14T08:39:46.658Z
6565d682995cc49553fa9f14
/avatars/0b85c4f9316e2e8c9fea37a0df10089d.svg
Hongjie Li
user
awfuact
65f27d5bdca94df6bdf9b8a0
Xiaoxuan Ma
admin_assigned
2024-03-14T08:39:54.014Z
6569840044ce94a7016f0abb
/avatars/734a55c4077012701d5aff1510d440a2.svg
Xiaoxuan Ma
user
ShirleyMaxx
65f27d5bdca94df6bdf9b8a1
Zan Wang
claimed_verified
2024-03-14T12:31:51.219Z
63a30e84528eba15b13f7355
/avatars/79e26058c60fe0bc9f34f372b2e69ead.svg
Zan Wang
user
SuperZan
65f27d5bdca94df6bdf9b8a2
Yixin Chen
admin_assigned
2024-03-14T08:40:21.851Z
64c396def6fe448b1ad553d6
/avatars/b2ce4739f42dc00ee974fff7ee1cb301.svg
Yixin Chen
user
YixinChen
65f27d5bdca94df6bdf9b8a3
Tengyu Liu
admin_assigned
2024-03-14T08:40:30.501Z
634379a81bdd3dfa55dcbe82
https://cdn-avatars.huggingface.co/v1/production/uploads/1665366425318-noauth.png
Tengyu Liu
user
oxrider
65f27d5bdca94df6bdf9b8a4
Yixin Zhu
65f27d5bdca94df6bdf9b8a5
Siyuan Huang
admin_assigned
2024-03-14T08:41:32.006Z
63c7a33121bd95f80ed74652
/avatars/7dd59afea785a2bff0ec2b757abd474e.svg
Siyuan Huang
user
thuhsy
Scaling Up Dynamic Human-Scene Interaction Modeling
Confronting the challenges of data scarcity and advanced motion synthesis in
human-scene interaction modeling, we introduce the TRUMANS dataset alongside a
novel HSI motion synthesis method. TRUMANS stands as the most comprehensive
motion-captured HSI dataset currently available, encompassing over 15 hours of
human interactions across 100 indoor scenes. It intricately captures whole-body
human motions and part-level object dynamics, focusing on the realism of
contact. This dataset is further scaled up by transforming physical
environments into exact virtual models and applying extensive augmentations to
appearance and motion for both humans and objects while maintaining interaction
fidelity. Utilizing TRUMANS, we devise a diffusion-based autoregressive model
that efficiently generates HSI sequences of any length, taking into account
both scene context and intended actions. In experiments, our approach shows
remarkable zero-shot generalizability on a range of 3D scene datasets (e.g.,
PROX, Replica, ScanNet, ScanNet++), producing motions that closely mimic
original motion-captured sequences, as confirmed by quantitative experiments
and human studies.
A new diffusion-based autoregressive model using the TRUMANS dataset efficiently generates human-scene interaction motion sequences with high realism and zero-shot generalizability.
TRUMANS dataset diffusion-based autoregressive model human-scene interaction motion synthesis motion capture interaction fidelity scene context intended actions zero-shot generalizability PROX Replica ScanNet ScanNet++