title: InsActor: Instruction-driven Physics-based Characters
summary: Generating animation of physics-based characters with intuitive control has
  long been a desirable task with numerous applications. However, generating
  physically simulated animations that reflect high-level human instructions
  remains a difficult problem due to the complexity of physical environments and
  the richness of human language. In this paper, we present InsActor, a
  principled generative framework that leverages recent advancements in
  diffusion-based human motion models to produce instruction-driven animations of
  physics-based characters. Our framework empowers InsActor to capture complex
  relationships between high-level human instructions and character motions by
  employing diffusion policies for flexibly conditioned motion planning. To
  overcome invalid states and infeasible state transitions in planned motions,
  InsActor discovers low-level skills and maps plans to latent skill sequences in
  a compact latent space. Extensive experiments demonstrate that InsActor
  achieves state-of-the-art results on various tasks, including
  instruction-driven motion generation and instruction-driven waypoint heading.
  Notably, the ability of InsActor to generate physically simulated animations
  using high-level human instructions makes it a valuable tool, particularly in
  executing long-horizon tasks with a rich set of instructions.
paper_authors._id: 658e50bf4a24a387789af5e4
paper_authors.name: Jiawei Ren
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-29T09:00:19.773Z
paper_authors._id: 623f533a28672458f749b8e9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1651044799068-623f533a28672458f749b8e9.png
paper_authors.fullname: Jiawe Ren
paper_authors.type: user
paper_authors.user: jiawei011
paper_authors._id: 658e50bf4a24a387789af5e5
paper_authors.name: Mingyuan Zhang
paper_authors._id: 658e50bf4a24a387789af5e6
paper_authors.name: Cunjun Yu
paper_authors._id: 658e50bf4a24a387789af5e7
paper_authors.name: Xiao Ma
paper_authors._id: 658e50bf4a24a387789af5e8
paper_authors.name: Liang Pan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-05T16:13:01.142Z
paper_authors._id: 64ef30fb0de94d31b7921175
paper_authors.avatarUrl: /avatars/d4f4c7be460befc1cedec13bbf9db972.svg
paper_authors.fullname: pan
paper_authors.type: user
paper_authors.user: pldeqiushui
paper_authors._id: 658e50bf4a24a387789af5e9
paper_authors.name: Ziwei Liu
paper_title: InsActor: Instruction-driven Physics-based Characters
paper_summary: Generating animation of physics-based characters with intuitive control has
  long been a desirable task with numerous applications. However, generating
  physically simulated animations that reflect high-level human instructions
  remains a difficult problem due to the complexity of physical environments and
  the richness of human language. In this paper, we present InsActor, a
  principled generative framework that leverages recent advancements in
  diffusion-based human motion models to produce instruction-driven animations of
  physics-based characters. Our framework empowers InsActor to capture complex
  relationships between high-level human instructions and character motions by
  employing diffusion policies for flexibly conditioned motion planning. To
  overcome invalid states and infeasible state transitions in planned motions,
  InsActor discovers low-level skills and maps plans to latent skill sequences in
  a compact latent space. Extensive experiments demonstrate that InsActor
  achieves state-of-the-art results on various tasks, including
  instruction-driven motion generation and instruction-driven waypoint heading.
  Notably, the ability of InsActor to generate physically simulated animations
  using high-level human instructions makes it a valuable tool, particularly in
  executing long-horizon tasks with a rich set of instructions.
paper_ai_keywords: diffusion-based human motion models diffusion policies low-level skills latent space skill sequences instruction-driven motion generation instruction-driven waypoint heading physically simulated animations high-level human instructions