title: TokenHSI: Unified Synthesis of Physical Human-Scene Interactions through
    Task Tokenization
summary: Synthesizing diverse and physically plausible Human-Scene Interactions (HSI)
  is pivotal for both computer animation and embodied AI. Despite encouraging
  progress, current methods mainly focus on developing separate controllers, each
  specialized for a specific interaction task. This significantly hinders the
  ability to tackle a wide variety of challenging HSI tasks that require the
  integration of multiple skills, e.g., sitting down while carrying an object. To
  address this issue, we present TokenHSI, a single, unified transformer-based
  policy capable of multi-skill unification and flexible adaptation. The key
  insight is to model the humanoid proprioception as a separate shared token and
  combine it with distinct task tokens via a masking mechanism. Such a unified
  policy enables effective knowledge sharing across skills, thereby facilitating
  the multi-task training. Moreover, our policy architecture supports variable
  length inputs, enabling flexible adaptation of learned skills to new scenarios.
  By training additional task tokenizers, we can not only modify the geometries
  of interaction targets but also coordinate multiple skills to address complex
  tasks. The experiments demonstrate that our approach can significantly improve
  versatility, adaptability, and extensibility in various HSI tasks. Website:
  https://liangpan99.github.io/TokenHSI/
paper_authors._id: 67eac6433755a17e3cbff585
paper_authors.name: Liang Pan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-01T07:47:34.754Z
paper_authors._id: 6630cc7e9ee8861dd0b9bdbd
paper_authors.avatarUrl: /avatars/01da7d60cbb107d58329bbb80d924eb4.svg
paper_authors.fullname: Liang Pan
paper_authors.type: user
paper_authors.user: lianganimation
paper_authors._id: 67eac6433755a17e3cbff586
paper_authors.name: Zeshi Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-01T08:21:03.851Z
paper_authors._id: 649c178950b4be74229d680f
paper_authors.avatarUrl: /avatars/941dec90fdfc46b9ae23378e3a3113f4.svg
paper_authors.fullname: Zeshi Yang
paper_authors.type: user
paper_authors.user: Zeshi209
paper_authors._id: 67eac6433755a17e3cbff587
paper_authors.name: Zhiyang Dou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-01T08:21:09.328Z
paper_authors._id: 645223fb01d7bd9555ea399a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/645223fb01d7bd9555ea399a/fVR7XmGg6pMSRKx9sEdvT.png
paper_authors.fullname: Zhiyang Dou
paper_authors.type: user
paper_authors.user: frankzydou
paper_authors._id: 67eac6433755a17e3cbff588
paper_authors.name: Wenjia Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-01T08:21:22.428Z
paper_authors._id: 6437a813fac5ea753f1c72d2
paper_authors.avatarUrl: /avatars/69e60e60497e404149a1dad46649dad4.svg
paper_authors.fullname: wenjia Wang
paper_authors.type: user
paper_authors.user: WenjiaWang
paper_authors._id: 67eac6433755a17e3cbff589
paper_authors.name: Buzhen Huang
paper_authors._id: 67eac6433755a17e3cbff58a
paper_authors.name: Bo Dai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-01T08:21:59.939Z
paper_authors._id: 635f93577c05eb9f59966209
paper_authors.avatarUrl: /avatars/add48d7e3790a6b500d6c451ef8b0f75.svg
paper_authors.fullname: Intelligent Digital Creation
paper_authors.type: user
paper_authors.user: BoDai
paper_authors._id: 67eac6433755a17e3cbff58b
paper_authors.name: Taku Komura
paper_authors._id: 67eac6433755a17e3cbff58c
paper_authors.name: Jingbo Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-01T08:21:43.455Z
paper_authors._id: 669cb638666901f41dae51bf
paper_authors.avatarUrl: /avatars/a7cc19e3db84bd86bee3eb6fd4897959.svg
paper_authors.fullname: Jingbo Wang
paper_authors.type: user
paper_authors.user: jingbocuhk
paper_title: TokenHSI: Unified Synthesis of Physical Human-Scene Interactions through
    Task Tokenization
paper_summary: Synthesizing diverse and physically plausible Human-Scene Interactions (HSI)
  is pivotal for both computer animation and embodied AI. Despite encouraging
  progress, current methods mainly focus on developing separate controllers, each
  specialized for a specific interaction task. This significantly hinders the
  ability to tackle a wide variety of challenging HSI tasks that require the
  integration of multiple skills, e.g., sitting down while carrying an object. To
  address this issue, we present TokenHSI, a single, unified transformer-based
  policy capable of multi-skill unification and flexible adaptation. The key
  insight is to model the humanoid proprioception as a separate shared token and
  combine it with distinct task tokens via a masking mechanism. Such a unified
  policy enables effective knowledge sharing across skills, thereby facilitating
  the multi-task training. Moreover, our policy architecture supports variable
  length inputs, enabling flexible adaptation of learned skills to new scenarios.
  By training additional task tokenizers, we can not only modify the geometries
  of interaction targets but also coordinate multiple skills to address complex
  tasks. The experiments demonstrate that our approach can significantly improve
  versatility, adaptability, and extensibility in various HSI tasks. Website:
  https://liangpan99.github.io/TokenHSI/
paper_ai_summary: A unified transformer-based policy, TokenHSI, improves versatility in Human-Scene Interactions by integrating multiple interaction skills through shared and task-specific tokens.
paper_ai_keywords: transformer-based policy TokenHSI humanoid proprioception shared token task tokens masking mechanism multi-task training variable length inputs