title: FlexiAct: Towards Flexible Action Control in Heterogeneous Scenarios
summary: Action customization involves generating videos where the subject performs
  actions dictated by input control signals. Current methods use pose-guided or
  global motion customization but are limited by strict constraints on spatial
  structure, such as layout, skeleton, and viewpoint consistency, reducing
  adaptability across diverse subjects and scenarios. To overcome these
  limitations, we propose FlexiAct, which transfers actions from a reference
  video to an arbitrary target image. Unlike existing methods, FlexiAct allows
  for variations in layout, viewpoint, and skeletal structure between the subject
  of the reference video and the target image, while maintaining identity
  consistency. Achieving this requires precise action control, spatial structure
  adaptation, and consistency preservation. To this end, we introduce RefAdapter,
  a lightweight image-conditioned adapter that excels in spatial adaptation and
  consistency preservation, surpassing existing methods in balancing appearance
  consistency and structural flexibility. Additionally, based on our
  observations, the denoising process exhibits varying levels of attention to
  motion (low frequency) and appearance details (high frequency) at different
  timesteps. So we propose FAE (Frequency-aware Action Extraction), which, unlike
  existing methods that rely on separate spatial-temporal architectures, directly
  achieves action extraction during the denoising process. Experiments
  demonstrate that our method effectively transfers actions to subjects with
  diverse layouts, skeletons, and viewpoints. We release our code and model
  weights to support further research at
  https://shiyi-zh0408.github.io/projectpages/FlexiAct/
paper_authors._id: 681abf3759155282c1cb2306
paper_authors.name: Shiyi Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-09T08:55:32.244Z
paper_authors._id: 6315d306a9456afe2b9bf34a
paper_authors.avatarUrl: /avatars/7285b4e7d84b528d1a50f8ee4eb10727.svg
paper_authors.fullname: ElevenZ
paper_authors.type: user
paper_authors.user: shiyi0408
paper_authors._id: 681abf3759155282c1cb2307
paper_authors.name: Junhao Zhuang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-07T08:08:58.214Z
paper_authors._id: 64970d3d9c3b29dca8633f87
paper_authors.avatarUrl: /avatars/11e3c9c66d28490d6d09925f9aa47cd1.svg
paper_authors.fullname: JunhaoZhuang
paper_authors.type: user
paper_authors.user: JunhaoZhuang
paper_authors._id: 681abf3759155282c1cb2308
paper_authors.name: Zhaoyang Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-08T08:59:37.193Z
paper_authors._id: 658409ceca19ccf6d9989add
paper_authors.avatarUrl: /avatars/3ac1dbd25e52435185babdeb3da28875.svg
paper_authors.fullname: Zhaoyang Zhang
paper_authors.type: user
paper_authors.user: ZyZcuhk
paper_authors._id: 681abf3759155282c1cb2309
paper_authors.name: Ying Shan
paper_authors._id: 681abf3759155282c1cb230a
paper_authors.name: Yansong Tang
paper_title: FlexiAct: Towards Flexible Action Control in Heterogeneous Scenarios
paper_summary: Action customization involves generating videos where the subject performs
  actions dictated by input control signals. Current methods use pose-guided or
  global motion customization but are limited by strict constraints on spatial
  structure, such as layout, skeleton, and viewpoint consistency, reducing
  adaptability across diverse subjects and scenarios. To overcome these
  limitations, we propose FlexiAct, which transfers actions from a reference
  video to an arbitrary target image. Unlike existing methods, FlexiAct allows
  for variations in layout, viewpoint, and skeletal structure between the subject
  of the reference video and the target image, while maintaining identity
  consistency. Achieving this requires precise action control, spatial structure
  adaptation, and consistency preservation. To this end, we introduce RefAdapter,
  a lightweight image-conditioned adapter that excels in spatial adaptation and
  consistency preservation, surpassing existing methods in balancing appearance
  consistency and structural flexibility. Additionally, based on our
  observations, the denoising process exhibits varying levels of attention to
  motion (low frequency) and appearance details (high frequency) at different
  timesteps. So we propose FAE (Frequency-aware Action Extraction), which, unlike
  existing methods that rely on separate spatial-temporal architectures, directly
  achieves action extraction during the denoising process. Experiments
  demonstrate that our method effectively transfers actions to subjects with
  diverse layouts, skeletons, and viewpoints. We release our code and model
  weights to support further research at
  https://shiyi-zh0408.github.io/projectpages/FlexiAct/
paper_ai_summary: FlexiAct transfers actions from reference videos to arbitrary target images, adapting to diverse layouts, viewpoints, and skeletal structures while maintaining identity consistency using RefAdapter and FAE.
paper_ai_keywords: pose-guided global motion customization FlexiAct RefAdapter image-conditioned adapter frequency-aware action extraction FAE denoising process action control spatial structure adaptation consistency preservation