WalkTheDog: Cross-Morphology Motion Alignment via Phase Manifolds
We present a new approach for understanding the periodicity structure and
semantics of motion datasets, independently of the morphology and skeletal
structure of characters. Unlike existing methods using an overly sparse
high-dimensional latent, we propose a phase manifold consisting of multiple
closed curves, each corresponding to a latent amplitude. With our proposed
vector quantized periodic autoencoder, we learn a shared phase manifold for
multiple characters, such as a human and a dog, without any supervision. This
is achieved by exploiting the discrete structure and a shallow network as
bottlenecks, such that semantically similar motions are clustered into the same
curve of the manifold, and the motions within the same component are aligned
temporally by the phase variable. In combination with an improved motion
matching framework, we demonstrate the manifold's capability of timing and
semantics alignment in several applications, including motion retrieval,
transfer and stylization. Code and pre-trained models for this paper are
available at https://peizhuoli.github.io/walkthedog.
66a8512d45f5642d5f02f8b1
Peizhuo Li
claimed_verified
2024-07-30T13:46:22.874Z
66a8e7d4a203add9773fe968
/avatars/0f42c55a02aef4bcca64f774fbd23c15.svg
Peizhuo Li
user
PeizhuoL
66a8512d45f5642d5f02f8b2
Sebastian Starke
66a8512d45f5642d5f02f8b3
Yuting Ye
66a8512d45f5642d5f02f8b4
Olga Sorkine-Hornung
WalkTheDog: Cross-Morphology Motion Alignment via Phase Manifolds
We present a new approach for understanding the periodicity structure and
semantics of motion datasets, independently of the morphology and skeletal
structure of characters. Unlike existing methods using an overly sparse
high-dimensional latent, we propose a phase manifold consisting of multiple
closed curves, each corresponding to a latent amplitude. With our proposed
vector quantized periodic autoencoder, we learn a shared phase manifold for
multiple characters, such as a human and a dog, without any supervision. This
is achieved by exploiting the discrete structure and a shallow network as
bottlenecks, such that semantically similar motions are clustered into the same
curve of the manifold, and the motions within the same component are aligned
temporally by the phase variable. In combination with an improved motion
matching framework, we demonstrate the manifold's capability of timing and
semantics alignment in several applications, including motion retrieval,
transfer and stylization. Code and pre-trained models for this paper are
available at https://peizhuoli.github.io/walkthedog.
A vector quantized periodic autoencoder learns a shared phase manifold for character motion, enabling semantic and temporal alignment in various applications.
phase manifold vector quantized periodic autoencoder latent amplitude shallow network motion retrieval motion transfer motion stylization