FlexiAct: Towards Flexible Action Control in Heterogeneous Scenarios
Action customization involves generating videos where the subject performs
actions dictated by input control signals. Current methods use pose-guided or
global motion customization but are limited by strict constraints on spatial
structure, such as layout, skeleton, and viewpoint consistency, reducing
adaptability across diverse subjects and scenarios. To overcome these
limitations, we propose FlexiAct, which transfers actions from a reference
video to an arbitrary target image. Unlike existing methods, FlexiAct allows
for variations in layout, viewpoint, and skeletal structure between the subject
of the reference video and the target image, while maintaining identity
consistency. Achieving this requires precise action control, spatial structure
adaptation, and consistency preservation. To this end, we introduce RefAdapter,
a lightweight image-conditioned adapter that excels in spatial adaptation and
consistency preservation, surpassing existing methods in balancing appearance
consistency and structural flexibility. Additionally, based on our
observations, the denoising process exhibits varying levels of attention to
motion (low frequency) and appearance details (high frequency) at different
timesteps. So we propose FAE (Frequency-aware Action Extraction), which, unlike
existing methods that rely on separate spatial-temporal architectures, directly
achieves action extraction during the denoising process. Experiments
demonstrate that our method effectively transfers actions to subjects with
diverse layouts, skeletons, and viewpoints. We release our code and model
weights to support further research at
https://shiyi-zh0408.github.io/projectpages/FlexiAct/
681abf3759155282c1cb2306
Shiyi Zhang
claimed_verified
2025-05-09T08:55:32.244Z
6315d306a9456afe2b9bf34a
/avatars/7285b4e7d84b528d1a50f8ee4eb10727.svg
ElevenZ
user
shiyi0408
681abf3759155282c1cb2307
Junhao Zhuang
claimed_verified
2025-05-07T08:08:58.214Z
64970d3d9c3b29dca8633f87
/avatars/11e3c9c66d28490d6d09925f9aa47cd1.svg
JunhaoZhuang
user
JunhaoZhuang
681abf3759155282c1cb2308
Zhaoyang Zhang
claimed_verified
2025-05-08T08:59:37.193Z
658409ceca19ccf6d9989add
/avatars/3ac1dbd25e52435185babdeb3da28875.svg
Zhaoyang Zhang
user
ZyZcuhk
681abf3759155282c1cb2309
Ying Shan
681abf3759155282c1cb230a
Yansong Tang
FlexiAct: Towards Flexible Action Control in Heterogeneous Scenarios
Action customization involves generating videos where the subject performs
actions dictated by input control signals. Current methods use pose-guided or
global motion customization but are limited by strict constraints on spatial
structure, such as layout, skeleton, and viewpoint consistency, reducing
adaptability across diverse subjects and scenarios. To overcome these
limitations, we propose FlexiAct, which transfers actions from a reference
video to an arbitrary target image. Unlike existing methods, FlexiAct allows
for variations in layout, viewpoint, and skeletal structure between the subject
of the reference video and the target image, while maintaining identity
consistency. Achieving this requires precise action control, spatial structure
adaptation, and consistency preservation. To this end, we introduce RefAdapter,
a lightweight image-conditioned adapter that excels in spatial adaptation and
consistency preservation, surpassing existing methods in balancing appearance
consistency and structural flexibility. Additionally, based on our
observations, the denoising process exhibits varying levels of attention to
motion (low frequency) and appearance details (high frequency) at different
timesteps. So we propose FAE (Frequency-aware Action Extraction), which, unlike
existing methods that rely on separate spatial-temporal architectures, directly
achieves action extraction during the denoising process. Experiments
demonstrate that our method effectively transfers actions to subjects with
diverse layouts, skeletons, and viewpoints. We release our code and model
weights to support further research at
https://shiyi-zh0408.github.io/projectpages/FlexiAct/
FlexiAct transfers actions from reference videos to arbitrary target images, adapting to diverse layouts, viewpoints, and skeletal structures while maintaining identity consistency using RefAdapter and FAE.
pose-guided global motion customization FlexiAct RefAdapter image-conditioned adapter frequency-aware action extraction FAE denoising process action control spatial structure adaptation consistency preservation