title: MaskedMimic: Unified Physics-Based Character Control Through Masked
    Motion Inpainting
summary: Crafting a single, versatile physics-based controller that can breathe life
  into interactive characters across a wide spectrum of scenarios represents an
  exciting frontier in character animation. An ideal controller should support
  diverse control modalities, such as sparse target keyframes, text instructions,
  and scene information. While previous works have proposed physically simulated,
  scene-aware control models, these systems have predominantly focused on
  developing controllers that each specializes in a narrow set of tasks and
  control modalities. This work presents MaskedMimic, a novel approach that
  formulates physics-based character control as a general motion inpainting
  problem. Our key insight is to train a single unified model to synthesize
  motions from partial (masked) motion descriptions, such as masked keyframes,
  objects, text descriptions, or any combination thereof. This is achieved by
  leveraging motion tracking data and designing a scalable training method that
  can effectively utilize diverse motion descriptions to produce coherent
  animations. Through this process, our approach learns a physics-based
  controller that provides an intuitive control interface without requiring
  tedious reward engineering for all behaviors of interest. The resulting
  controller supports a wide range of control modalities and enables seamless
  transitions between disparate tasks. By unifying character control through
  motion inpainting, MaskedMimic creates versatile virtual characters. These
  characters can dynamically adapt to complex scenes and compose diverse motions
  on demand, enabling more interactive and immersive experiences.
paper_authors._id: 66f239eb50a4aeef226d2733
paper_authors.name: Chen Tessler
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-03-18T21:31:43.194Z
paper_authors._id: 67d9db8a8d942645bf063484
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/7JeFDO4Ky7XjTCRL4ROBl.png
paper_authors.fullname: Chen Tessler
paper_authors.type: user
paper_authors.user: ctessler
paper_authors._id: 66f239eb50a4aeef226d2734
paper_authors.name: Yunrong Guo
paper_authors._id: 66f239eb50a4aeef226d2735
paper_authors.name: Ofir Nabati
paper_authors._id: 66f239eb50a4aeef226d2736
paper_authors.name: Gal Chechik
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-09-24T04:02:54.966Z
paper_authors._id: 6493393f357b252af72196c5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6493393f357b252af72196c5/EWSy18XRcMRa_4XMM3Fu-.jpeg
paper_authors.fullname: Gal Chechik
paper_authors.type: user
paper_authors.user: galchechik
paper_authors._id: 66f239eb50a4aeef226d2737
paper_authors.name: Xue Bin Peng
paper_title: MaskedMimic: Unified Physics-Based Character Control Through Masked
    Motion Inpainting
paper_summary: Crafting a single, versatile physics-based controller that can breathe life
  into interactive characters across a wide spectrum of scenarios represents an
  exciting frontier in character animation. An ideal controller should support
  diverse control modalities, such as sparse target keyframes, text instructions,
  and scene information. While previous works have proposed physically simulated,
  scene-aware control models, these systems have predominantly focused on
  developing controllers that each specializes in a narrow set of tasks and
  control modalities. This work presents MaskedMimic, a novel approach that
  formulates physics-based character control as a general motion inpainting
  problem. Our key insight is to train a single unified model to synthesize
  motions from partial (masked) motion descriptions, such as masked keyframes,
  objects, text descriptions, or any combination thereof. This is achieved by
  leveraging motion tracking data and designing a scalable training method that
  can effectively utilize diverse motion descriptions to produce coherent
  animations. Through this process, our approach learns a physics-based
  controller that provides an intuitive control interface without requiring
  tedious reward engineering for all behaviors of interest. The resulting
  controller supports a wide range of control modalities and enables seamless
  transitions between disparate tasks. By unifying character control through
  motion inpainting, MaskedMimic creates versatile virtual characters. These
  characters can dynamically adapt to complex scenes and compose diverse motions
  on demand, enabling more interactive and immersive experiences.
paper_ai_keywords: physics-based controller control modalities sparse target keyframes text instructions scene-aware control models MaskedMimic general motion inpainting partial (masked) motion descriptions masked keyframes motion tracking data scalable training method coherent animations reward engineering intuitive control interface versatile virtual characters complex scenes diverse motions interactive experiences