title: MotionSight: Boosting Fine-Grained Motion Understanding in Multimodal
    LLMs
summary: Despite advancements in Multimodal Large Language Models (MLLMs), their
  proficiency in fine-grained video motion understanding remains critically
  limited. They often lack inter-frame differencing and tend to average or ignore
  subtle visual cues. Furthermore, while visual prompting has shown potential in
  static images, its application to video's temporal complexities, particularly
  for fine-grained motion understanding, remains largely unexplored. We
  investigate whether inherent capability can be unlocked and boost MLLMs' motion
  perception and enable distinct visual signatures tailored to decouple object
  and camera motion cues. In this study, we introduce MotionSight, a novel
  zero-shot method pioneering object-centric visual spotlight and motion blur as
  visual prompts to effectively improve fine-grained motion understanding without
  training. To convert this into valuable data assets, we curated MotionVid-QA,
  the first large-scale dataset for fine-grained video motion understanding, with
  hierarchical annotations including SFT and preference data, {\Theta}(40K) video
  clips and {\Theta}(87K) QAs. Experiments show MotionSight achieves
  state-of-the-art open-source performance and competitiveness with commercial
  models. In particular, for fine-grained motion understanding we present a novel
  zero-shot technique and a large-scale, high-quality dataset. All the code and
  annotations will be publicly available.
paper_authors._id: 683faa6515abeae85e13336b
paper_authors.name: Yipeng Du
paper_authors._id: 683faa6515abeae85e13336c
paper_authors.name: Tiehan Fan
paper_authors._id: 683faa6515abeae85e13336d
paper_authors.name: Kepan Nan
paper_authors._id: 683faa6515abeae85e13336e
paper_authors.name: Rui Xie
paper_authors._id: 683faa6515abeae85e13336f
paper_authors.name: Penghao Zhou
paper_authors._id: 683faa6515abeae85e133370
paper_authors.name: Xiang Li
paper_authors._id: 683faa6515abeae85e133371
paper_authors.name: Jian Yang
paper_authors._id: 683faa6515abeae85e133372
paper_authors.name: Zhenheng Yang
paper_authors._id: 683faa6515abeae85e133373
paper_authors.name: Ying Tai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T08:56:59.253Z
paper_authors._id: 65734004769f3ee9bde1af10
paper_authors.avatarUrl: /avatars/d6310ed861972fd691687d8f47413f33.svg
paper_authors.fullname: Ying Tai
paper_authors.type: user
paper_authors.user: yingtai
paper_title: MotionSight: Boosting Fine-Grained Motion Understanding in Multimodal
    LLMs
paper_summary: Despite advancements in Multimodal Large Language Models (MLLMs), their
  proficiency in fine-grained video motion understanding remains critically
  limited. They often lack inter-frame differencing and tend to average or ignore
  subtle visual cues. Furthermore, while visual prompting has shown potential in
  static images, its application to video's temporal complexities, particularly
  for fine-grained motion understanding, remains largely unexplored. We
  investigate whether inherent capability can be unlocked and boost MLLMs' motion
  perception and enable distinct visual signatures tailored to decouple object
  and camera motion cues. In this study, we introduce MotionSight, a novel
  zero-shot method pioneering object-centric visual spotlight and motion blur as
  visual prompts to effectively improve fine-grained motion understanding without
  training. To convert this into valuable data assets, we curated MotionVid-QA,
  the first large-scale dataset for fine-grained video motion understanding, with
  hierarchical annotations including SFT and preference data, {\Theta}(40K) video
  clips and {\Theta}(87K) QAs. Experiments show MotionSight achieves
  state-of-the-art open-source performance and competitiveness with commercial
  models. In particular, for fine-grained motion understanding we present a novel
  zero-shot technique and a large-scale, high-quality dataset. All the code and
  annotations will be publicly available.
paper_ai_summary: MotionSight, a zero-shot method using object-centric visual spotlight and motion blur as prompts, enhances fine-grained video motion understanding and achieves state-of-the-art performance on MotionVid-QA, a large-scale dataset with hierarchical annotations.
paper_ai_keywords: Multimodal Large Language Models MLLMs fine-grained video motion understanding inter-frame differencing visual prompting temporal complexities MotionSight object-centric visual spotlight motion blur MotionVid-QA hierarchical annotations SFT preference data state-of-the-art performance