title: Follow-Your-Click: Open-domain Regional Image Animation via Short
    Prompts
summary: Despite recent advances in image-to-video generation, better controllability
  and local animation are less explored. Most existing image-to-video methods are
  not locally aware and tend to move the entire scene. However, human artists may
  need to control the movement of different objects or regions. Additionally,
  current I2V methods require users not only to describe the target motion but
  also to provide redundant detailed descriptions of frame contents. These two
  issues hinder the practical utilization of current I2V tools. In this paper, we
  propose a practical framework, named Follow-Your-Click, to achieve image
  animation with a simple user click (for specifying what to move) and a short
  motion prompt (for specifying how to move). Technically, we propose the
  first-frame masking strategy, which significantly improves the video generation
  quality, and a motion-augmented module equipped with a short motion prompt
  dataset to improve the short prompt following abilities of our model. To
  further control the motion speed, we propose flow-based motion magnitude
  control to control the speed of target movement more precisely. Our framework
  has simpler yet precise user control and better generation performance than
  previous methods. Extensive experiments compared with 7 baselines, including
  both commercial tools and research methods on 8 metrics, suggest the
  superiority of our approach. Project Page: https://follow-your-click.github.io/
paper_authors._id: 65f2693333725f138b0f1719
paper_authors.name: Yue Ma
paper_authors._id: 65f2693333725f138b0f171a
paper_authors.name: Yingqing He
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:34:16.010Z
paper_authors._id: 630231bf7e137e3d6b3b0645
paper_authors.avatarUrl: /avatars/dad52f8955110f0a2caeb613d6aa3ea2.svg
paper_authors.fullname: He
paper_authors.type: user
paper_authors.user: Yingqing
paper_authors._id: 65f2693333725f138b0f171b
paper_authors.name: Hongfa Wang
paper_authors._id: 65f2693333725f138b0f171c
paper_authors.name: Andong Wang
paper_authors._id: 65f2693333725f138b0f171d
paper_authors.name: Chenyang Qi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:34:37.856Z
paper_authors._id: 642157a201c62c1e41fc526a
paper_authors.avatarUrl: /avatars/73ae831a73947a04468b1e87d17da24d.svg
paper_authors.fullname: chenyangqi
paper_authors.type: user
paper_authors.user: chenyangqi
paper_authors._id: 65f2693333725f138b0f171e
paper_authors.name: Chengfei Cai
paper_authors._id: 65f2693333725f138b0f171f
paper_authors.name: Xiu Li
paper_authors._id: 65f2693333725f138b0f1720
paper_authors.name: Zhifeng Li
paper_authors._id: 65f2693333725f138b0f1721
paper_authors.name: Heung-Yeung Shum
paper_authors._id: 65f2693333725f138b0f1722
paper_authors.name: Wei Liu
paper_authors._id: 65f2693333725f138b0f1723
paper_authors.name: Qifeng Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:35:20.475Z
paper_authors._id: 6467b121e7a6a374fd19b44b
paper_authors.avatarUrl: /avatars/3f2874d58986d651aef55e3408b05700.svg
paper_authors.fullname: Qifeng Chen
paper_authors.type: user
paper_authors.user: cqf
paper_title: Follow-Your-Click: Open-domain Regional Image Animation via Short
    Prompts
paper_summary: Despite recent advances in image-to-video generation, better controllability
  and local animation are less explored. Most existing image-to-video methods are
  not locally aware and tend to move the entire scene. However, human artists may
  need to control the movement of different objects or regions. Additionally,
  current I2V methods require users not only to describe the target motion but
  also to provide redundant detailed descriptions of frame contents. These two
  issues hinder the practical utilization of current I2V tools. In this paper, we
  propose a practical framework, named Follow-Your-Click, to achieve image
  animation with a simple user click (for specifying what to move) and a short
  motion prompt (for specifying how to move). Technically, we propose the
  first-frame masking strategy, which significantly improves the video generation
  quality, and a motion-augmented module equipped with a short motion prompt
  dataset to improve the short prompt following abilities of our model. To
  further control the motion speed, we propose flow-based motion magnitude
  control to control the speed of target movement more precisely. Our framework
  has simpler yet precise user control and better generation performance than
  previous methods. Extensive experiments compared with 7 baselines, including
  both commercial tools and research methods on 8 metrics, suggest the
  superiority of our approach. Project Page: https://follow-your-click.github.io/
paper_ai_summary: A framework named Follow-Your-Click allows precise image-to-video animation through user clicks and short motion prompts, enhancing controllability and local animation quality.
paper_ai_keywords: image-to-video generation local awareness first-frame masking motion-augmented module short motion prompt dataset flow-based motion magnitude control video generation quality