DragAnything: Motion Control for Anything using Entity Representation
We introduce DragAnything, which utilizes a entity representation to achieve
motion control for any object in controllable video generation. Comparison to
existing motion control methods, DragAnything offers several advantages.
Firstly, trajectory-based is more userfriendly for interaction, when acquiring
other guidance signals (e.g., masks, depth maps) is labor-intensive. Users only
need to draw a line (trajectory) during interaction. Secondly, our entity
representation serves as an open-domain embedding capable of representing any
object, enabling the control of motion for diverse entities, including
background. Lastly, our entity representation allows simultaneous and distinct
motion control for multiple objects. Extensive experiments demonstrate that our
DragAnything achieves state-of-the-art performance for FVD, FID, and User
Study, particularly in terms of object motion control, where our method
surpasses the previous methods (e.g., DragNUWA) by 26% in human voting.
65f10ccb3cf945cb5ad34297
Wejia Wu
claimed_verified
2024-03-13T16:15:04.656Z
6345a93afe134dfd7a0cfabd
/avatars/65130ce06b1c72ab1066678419731d88.svg
wu weijia
user
weijiawu
65f10ccb3cf945cb5ad34298
Zhuang Li
65f10ccb3cf945cb5ad34299
Yuchao Gu
admin_assigned
2024-03-13T14:08:45.518Z
63021630a35b21bd8a53305a
/avatars/7a7e8b39749eda61e57d8a1908726558.svg
Gu Yuchao
user
guyuchao
65f10ccb3cf945cb5ad3429a
Rui Zhao
admin_assigned
2024-03-13T14:09:05.553Z
652b83b73b5997ed71a310f2
https://cdn-avatars.huggingface.co/v1/production/uploads/652b83b73b5997ed71a310f2/ipCpdeHUp4-0OmRz5z8IW.png
Rui Zhao
user
ruizhaocv
65f10ccb3cf945cb5ad3429b
Yefei He
admin_assigned
2024-03-13T14:09:16.095Z
65a88c3d26598b995531fff1
/avatars/b1a524857d8572d0405476661b434160.svg
Yefei He
user
yefly
65f10ccb3cf945cb5ad3429c
David Junhao Zhang
admin_assigned
2024-03-13T14:09:22.678Z
6468855763a564ba347e1efa
/avatars/caa99c3318c1d6d21be44d2d32795a62.svg
David Junhao ZHANG
user
Junhao233
65f10ccb3cf945cb5ad3429d
Mike Zheng Shou
admin_assigned
2024-03-13T14:09:46.187Z
63a55320ce5763e06f78519c
https://cdn-avatars.huggingface.co/v1/production/uploads/1671779060549-noauth.jpeg
Mike Shou
user
mikeshou
65f10ccb3cf945cb5ad3429e
Yan Li
65f10ccb3cf945cb5ad3429f
Tingting Gao
65f10ccb3cf945cb5ad342a0
Di Zhang
DragAnything: Motion Control for Anything using Entity Representation
We introduce DragAnything, which utilizes a entity representation to achieve
motion control for any object in controllable video generation. Comparison to
existing motion control methods, DragAnything offers several advantages.
Firstly, trajectory-based is more userfriendly for interaction, when acquiring
other guidance signals (e.g., masks, depth maps) is labor-intensive. Users only
need to draw a line (trajectory) during interaction. Secondly, our entity
representation serves as an open-domain embedding capable of representing any
object, enabling the control of motion for diverse entities, including
background. Lastly, our entity representation allows simultaneous and distinct
motion control for multiple objects. Extensive experiments demonstrate that our
DragAnything achieves state-of-the-art performance for FVD, FID, and User
Study, particularly in terms of object motion control, where our method
surpasses the previous methods (e.g., DragNUWA) by 26% in human voting.
DragAnything uses entity representation for motion control in video generation, offering user-friendly interaction and distinct motion control for multiple objects.
entity representation trajectory-based motion control video generation FVD FID user study DragNUWA