Animate-X: Universal Character Image Animation with Enhanced Motion
  Representation
Character image animation, which generates high-quality videos from a
reference image and target pose sequence, has seen significant progress in
recent years. However, most existing methods only apply to human figures, which
usually do not generalize well on anthropomorphic characters commonly used in
industries like gaming and entertainment. Our in-depth analysis suggests to
attribute this limitation to their insufficient modeling of motion, which is
unable to comprehend the movement pattern of the driving video, thus imposing a
pose sequence rigidly onto the target character. To this end, this paper
proposes Animate-X, a universal animation framework based on LDM for various
character types (collectively named X), including anthropomorphic characters.
To enhance motion representation, we introduce the Pose Indicator, which
captures comprehensive motion pattern from the driving video through both
implicit and explicit manner. The former leverages CLIP visual features of a
driving video to extract its gist of motion, like the overall movement pattern
and temporal relations among motions, while the latter strengthens the
generalization of LDM by simulating possible inputs in advance that may arise
during inference. Moreover, we introduce a new Animated Anthropomorphic
Benchmark (A^2Bench) to evaluate the performance of Animate-X on universal and
widely applicable animation images. Extensive experiments demonstrate the
superiority and effectiveness of Animate-X compared to state-of-the-art
methods.
670dd7ee95c91805b9f5baaf
Shuai Tan
claimed_verified
2024-10-16T09:44:18.747Z
6697b7006fda0261942aba22
/avatars/c54f25254eaf0cf454a97a86e85af035.svg
Shuai Tan
user
tanshuai0219
670dd7ee95c91805b9f5bab0
Biao Gong
claimed_verified
2024-10-15T09:10:56.446Z
644fcbea4f7316588267dc80
https://cdn-avatars.huggingface.co/v1/production/uploads/644fcbea4f7316588267dc80/w8-2Gkaw9BN9VzppNXrTP.jpeg
Biao Gong
user
BiaoGong
670dd7ee95c91805b9f5bab1
Xiang Wang
670dd7ee95c91805b9f5bab2
Shiwei Zhang
admin_assigned
2024-10-15T13:40:12.968Z
63a1200045edac9f7508bae9
/avatars/e84f0b045f32c5b8b4da43458650b925.svg
Shiwei Zhang
user
StevenZhang
670dd7ee95c91805b9f5bab3
Dandan Zheng
admin_assigned
2024-10-15T13:39:53.541Z
65dd699a89a2a760d15f7d35
/avatars/e098b56c413d147d1f38cf33a4b0ecde.svg
Dandan Zheng
user
zhengdd0422
670dd7ee95c91805b9f5bab4
Ruobing Zheng
670dd7ee95c91805b9f5bab5
Kecheng Zheng
claimed_verified
2024-10-28T11:25:44.171Z
64252045a4f3051f54dd1d53
/avatars/0e423a3291091be3b4736a14da3ce495.svg
kecheng zheng
user
zkcys001
670dd7ee95c91805b9f5bab6
Jingdong Chen
670dd7ee95c91805b9f5bab7
Ming Yang
Animate-X: Universal Character Image Animation with Enhanced Motion
  Representation
Character image animation, which generates high-quality videos from a
reference image and target pose sequence, has seen significant progress in
recent years. However, most existing methods only apply to human figures, which
usually do not generalize well on anthropomorphic characters commonly used in
industries like gaming and entertainment. Our in-depth analysis suggests to
attribute this limitation to their insufficient modeling of motion, which is
unable to comprehend the movement pattern of the driving video, thus imposing a
pose sequence rigidly onto the target character. To this end, this paper
proposes Animate-X, a universal animation framework based on LDM for various
character types (collectively named X), including anthropomorphic characters.
To enhance motion representation, we introduce the Pose Indicator, which
captures comprehensive motion pattern from the driving video through both
implicit and explicit manner. The former leverages CLIP visual features of a
driving video to extract its gist of motion, like the overall movement pattern
and temporal relations among motions, while the latter strengthens the
generalization of LDM by simulating possible inputs in advance that may arise
during inference. Moreover, we introduce a new Animated Anthropomorphic
Benchmark (A^2Bench) to evaluate the performance of Animate-X on universal and
widely applicable animation images. Extensive experiments demonstrate the
superiority and effectiveness of Animate-X compared to state-of-the-art
methods.
Animate-X, based on latent diffusion models, generates high-quality videos for various character types by enhancing motion representation through the Pose Indicator and a new benchmark, demonstrating superior performance.
LDM Animate-X Pose Indicator CLIP visual features Animated Anthropomorphic Benchmark (A^2Bench)