title: Animate-X: Universal Character Image Animation with Enhanced Motion
    Representation
summary: Character image animation, which generates high-quality videos from a
  reference image and target pose sequence, has seen significant progress in
  recent years. However, most existing methods only apply to human figures, which
  usually do not generalize well on anthropomorphic characters commonly used in
  industries like gaming and entertainment. Our in-depth analysis suggests to
  attribute this limitation to their insufficient modeling of motion, which is
  unable to comprehend the movement pattern of the driving video, thus imposing a
  pose sequence rigidly onto the target character. To this end, this paper
  proposes Animate-X, a universal animation framework based on LDM for various
  character types (collectively named X), including anthropomorphic characters.
  To enhance motion representation, we introduce the Pose Indicator, which
  captures comprehensive motion pattern from the driving video through both
  implicit and explicit manner. The former leverages CLIP visual features of a
  driving video to extract its gist of motion, like the overall movement pattern
  and temporal relations among motions, while the latter strengthens the
  generalization of LDM by simulating possible inputs in advance that may arise
  during inference. Moreover, we introduce a new Animated Anthropomorphic
  Benchmark (A^2Bench) to evaluate the performance of Animate-X on universal and
  widely applicable animation images. Extensive experiments demonstrate the
  superiority and effectiveness of Animate-X compared to state-of-the-art
  methods.
paper_authors._id: 670dd7ee95c91805b9f5baaf
paper_authors.name: Shuai Tan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-16T09:44:18.747Z
paper_authors._id: 6697b7006fda0261942aba22
paper_authors.avatarUrl: /avatars/c54f25254eaf0cf454a97a86e85af035.svg
paper_authors.fullname: Shuai Tan
paper_authors.type: user
paper_authors.user: tanshuai0219
paper_authors._id: 670dd7ee95c91805b9f5bab0
paper_authors.name: Biao Gong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-15T09:10:56.446Z
paper_authors._id: 644fcbea4f7316588267dc80
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/644fcbea4f7316588267dc80/w8-2Gkaw9BN9VzppNXrTP.jpeg
paper_authors.fullname: Biao Gong
paper_authors.type: user
paper_authors.user: BiaoGong
paper_authors._id: 670dd7ee95c91805b9f5bab1
paper_authors.name: Xiang Wang
paper_authors._id: 670dd7ee95c91805b9f5bab2
paper_authors.name: Shiwei Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-15T13:40:12.968Z
paper_authors._id: 63a1200045edac9f7508bae9
paper_authors.avatarUrl: /avatars/e84f0b045f32c5b8b4da43458650b925.svg
paper_authors.fullname: Shiwei Zhang
paper_authors.type: user
paper_authors.user: StevenZhang
paper_authors._id: 670dd7ee95c91805b9f5bab3
paper_authors.name: Dandan Zheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-15T13:39:53.541Z
paper_authors._id: 65dd699a89a2a760d15f7d35
paper_authors.avatarUrl: /avatars/e098b56c413d147d1f38cf33a4b0ecde.svg
paper_authors.fullname: Dandan Zheng
paper_authors.type: user
paper_authors.user: zhengdd0422
paper_authors._id: 670dd7ee95c91805b9f5bab4
paper_authors.name: Ruobing Zheng
paper_authors._id: 670dd7ee95c91805b9f5bab5
paper_authors.name: Kecheng Zheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-28T11:25:44.171Z
paper_authors._id: 64252045a4f3051f54dd1d53
paper_authors.avatarUrl: /avatars/0e423a3291091be3b4736a14da3ce495.svg
paper_authors.fullname: kecheng zheng
paper_authors.type: user
paper_authors.user: zkcys001
paper_authors._id: 670dd7ee95c91805b9f5bab6
paper_authors.name: Jingdong Chen
paper_authors._id: 670dd7ee95c91805b9f5bab7
paper_authors.name: Ming Yang
paper_title: Animate-X: Universal Character Image Animation with Enhanced Motion
    Representation
paper_summary: Character image animation, which generates high-quality videos from a
  reference image and target pose sequence, has seen significant progress in
  recent years. However, most existing methods only apply to human figures, which
  usually do not generalize well on anthropomorphic characters commonly used in
  industries like gaming and entertainment. Our in-depth analysis suggests to
  attribute this limitation to their insufficient modeling of motion, which is
  unable to comprehend the movement pattern of the driving video, thus imposing a
  pose sequence rigidly onto the target character. To this end, this paper
  proposes Animate-X, a universal animation framework based on LDM for various
  character types (collectively named X), including anthropomorphic characters.
  To enhance motion representation, we introduce the Pose Indicator, which
  captures comprehensive motion pattern from the driving video through both
  implicit and explicit manner. The former leverages CLIP visual features of a
  driving video to extract its gist of motion, like the overall movement pattern
  and temporal relations among motions, while the latter strengthens the
  generalization of LDM by simulating possible inputs in advance that may arise
  during inference. Moreover, we introduce a new Animated Anthropomorphic
  Benchmark (A^2Bench) to evaluate the performance of Animate-X on universal and
  widely applicable animation images. Extensive experiments demonstrate the
  superiority and effectiveness of Animate-X compared to state-of-the-art
  methods.
paper_ai_summary: Animate-X, based on latent diffusion models, generates high-quality videos for various character types by enhancing motion representation through the Pose Indicator and a new benchmark, demonstrating superior performance.
paper_ai_keywords: LDM Animate-X Pose Indicator CLIP visual features Animated Anthropomorphic Benchmark (A^2Bench)