MaskedMimic: Unified Physics-Based Character Control Through Masked
  Motion Inpainting
Crafting a single, versatile physics-based controller that can breathe life
into interactive characters across a wide spectrum of scenarios represents an
exciting frontier in character animation. An ideal controller should support
diverse control modalities, such as sparse target keyframes, text instructions,
and scene information. While previous works have proposed physically simulated,
scene-aware control models, these systems have predominantly focused on
developing controllers that each specializes in a narrow set of tasks and
control modalities. This work presents MaskedMimic, a novel approach that
formulates physics-based character control as a general motion inpainting
problem. Our key insight is to train a single unified model to synthesize
motions from partial (masked) motion descriptions, such as masked keyframes,
objects, text descriptions, or any combination thereof. This is achieved by
leveraging motion tracking data and designing a scalable training method that
can effectively utilize diverse motion descriptions to produce coherent
animations. Through this process, our approach learns a physics-based
controller that provides an intuitive control interface without requiring
tedious reward engineering for all behaviors of interest. The resulting
controller supports a wide range of control modalities and enables seamless
transitions between disparate tasks. By unifying character control through
motion inpainting, MaskedMimic creates versatile virtual characters. These
characters can dynamically adapt to complex scenes and compose diverse motions
on demand, enabling more interactive and immersive experiences.
66f239eb50a4aeef226d2733
Chen Tessler
extracted_confirmed
2025-03-18T21:31:43.194Z
67d9db8a8d942645bf063484
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/7JeFDO4Ky7XjTCRL4ROBl.png
Chen Tessler
user
ctessler
66f239eb50a4aeef226d2734
Yunrong Guo
66f239eb50a4aeef226d2735
Ofir Nabati
66f239eb50a4aeef226d2736
Gal Chechik
extracted_pending
2024-09-24T04:02:54.966Z
6493393f357b252af72196c5
https://cdn-avatars.huggingface.co/v1/production/uploads/6493393f357b252af72196c5/EWSy18XRcMRa_4XMM3Fu-.jpeg
Gal Chechik
user
galchechik
66f239eb50a4aeef226d2737
Xue Bin Peng
MaskedMimic: Unified Physics-Based Character Control Through Masked
  Motion Inpainting
Crafting a single, versatile physics-based controller that can breathe life
into interactive characters across a wide spectrum of scenarios represents an
exciting frontier in character animation. An ideal controller should support
diverse control modalities, such as sparse target keyframes, text instructions,
and scene information. While previous works have proposed physically simulated,
scene-aware control models, these systems have predominantly focused on
developing controllers that each specializes in a narrow set of tasks and
control modalities. This work presents MaskedMimic, a novel approach that
formulates physics-based character control as a general motion inpainting
problem. Our key insight is to train a single unified model to synthesize
motions from partial (masked) motion descriptions, such as masked keyframes,
objects, text descriptions, or any combination thereof. This is achieved by
leveraging motion tracking data and designing a scalable training method that
can effectively utilize diverse motion descriptions to produce coherent
animations. Through this process, our approach learns a physics-based
controller that provides an intuitive control interface without requiring
tedious reward engineering for all behaviors of interest. The resulting
controller supports a wide range of control modalities and enables seamless
transitions between disparate tasks. By unifying character control through
motion inpainting, MaskedMimic creates versatile virtual characters. These
characters can dynamically adapt to complex scenes and compose diverse motions
on demand, enabling more interactive and immersive experiences.
MaskedMimic unifies physics-based character control by formulating it as a motion inpainting problem, enabling versatile and intuitive character animation across various modalities and tasks.
physics-based character control motion inpainting masked keyframes text descriptions motion tracking data scalable training method physics-based controller character animation intuitive control interface task transitions versatile virtual characters scene adaptation diverse motions interactive experience