title: MotionBench: Benchmarking and Improving Fine-grained Video Motion
    Understanding for Vision Language Models
summary: In recent years, vision language models (VLMs) have made significant
  advancements in video understanding. However, a crucial capability -
  fine-grained motion comprehension - remains under-explored in current
  benchmarks. To address this gap, we propose MotionBench, a comprehensive
  evaluation benchmark designed to assess the fine-grained motion comprehension
  of video understanding models. MotionBench evaluates models' motion-level
  perception through six primary categories of motion-oriented question types and
  includes data collected from diverse sources, ensuring a broad representation
  of real-world video content. Experimental results reveal that existing VLMs
  perform poorly in understanding fine-grained motions. To enhance VLM's ability
  to perceive fine-grained motion within a limited sequence length of LLM, we
  conduct extensive experiments reviewing VLM architectures optimized for video
  feature compression and propose a novel and efficient Through-Encoder (TE)
  Fusion method. Experiments show that higher frame rate inputs and TE Fusion
  yield improvements in motion understanding, yet there is still substantial room
  for enhancement. Our benchmark aims to guide and motivate the development of
  more capable video understanding models, emphasizing the importance of
  fine-grained motion comprehension. Project page: https://motion-bench.github.io .
paper_authors._id: 677dfb5d0310e9426191dd3e
paper_authors.name: Wenyi Hong
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-01-08T04:13:19.132Z
paper_authors._id: 62ecd24cb8764c7738ef2793
paper_authors.avatarUrl: /avatars/c1b80b5c55f9d652c1aaac7919e1fa32.svg
paper_authors.fullname: Wenyi Hong
paper_authors.type: user
paper_authors.user: wenyi
paper_authors._id: 677dfb5d0310e9426191dd3f
paper_authors.name: Yean Cheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-08T09:44:31.589Z
paper_authors._id: 65acc5afe2a2c8635614de43
paper_authors.avatarUrl: /avatars/c5fce792792cc0b52ed7475d72460c58.svg
paper_authors.fullname: Yean Cheng
paper_authors.type: user
paper_authors.user: LiquidAmmonia
paper_authors._id: 677dfb5d0310e9426191dd40
paper_authors.name: Zhuoyi Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-08T13:26:08.664Z
paper_authors._id: 6466d1640ed2f7a8cba87503
paper_authors.avatarUrl: /avatars/652746e63dfeb5154ae7d34039d1a485.svg
paper_authors.fullname: Zhuoyi Yang
paper_authors.type: user
paper_authors.user: zyyangzy
paper_authors._id: 677dfb5d0310e9426191dd41
paper_authors.name: Weihan Wang
paper_authors._id: 677dfb5d0310e9426191dd42
paper_authors.name: Lefan Wang
paper_authors._id: 677dfb5d0310e9426191dd43
paper_authors.name: Xiaotao Gu
paper_authors._id: 677dfb5d0310e9426191dd44
paper_authors.name: Shiyu Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-08T13:27:03.140Z
paper_authors._id: 6406db5cd684369027166986
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6406db5cd684369027166986/Zl-orrGcbY0RbfjfKszn1.jpeg
paper_authors.fullname: Shiyu Huang
paper_authors.type: user
paper_authors.user: ShiyuHuang
paper_authors._id: 677dfb5d0310e9426191dd45
paper_authors.name: Yuxiao Dong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-08T13:27:12.264Z
paper_authors._id: 640e73bdfdeaae1390857b62
paper_authors.avatarUrl: /avatars/cd6779e30f716002a7838ed93d5c0754.svg
paper_authors.fullname: Yuxiao Dong
paper_authors.type: user
paper_authors.user: yuxiaod
paper_authors._id: 677dfb5d0310e9426191dd46
paper_authors.name: Jie Tang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-08T13:27:19.977Z
paper_authors._id: 640dff05474aa6f89556677e
paper_authors.avatarUrl: /avatars/1b4591c7322d649c797b3125148f1915.svg
paper_authors.fullname: Jie Tang
paper_authors.type: user
paper_authors.user: jerytang
paper_title: MotionBench: Benchmarking and Improving Fine-grained Video Motion
    Understanding for Vision Language Models
paper_summary: In recent years, vision language models (VLMs) have made significant
  advancements in video understanding. However, a crucial capability -
  fine-grained motion comprehension - remains under-explored in current
  benchmarks. To address this gap, we propose MotionBench, a comprehensive
  evaluation benchmark designed to assess the fine-grained motion comprehension
  of video understanding models. MotionBench evaluates models' motion-level
  perception through six primary categories of motion-oriented question types and
  includes data collected from diverse sources, ensuring a broad representation
  of real-world video content. Experimental results reveal that existing VLMs
  perform poorly in understanding fine-grained motions. To enhance VLM's ability
  to perceive fine-grained motion within a limited sequence length of LLM, we
  conduct extensive experiments reviewing VLM architectures optimized for video
  feature compression and propose a novel and efficient Through-Encoder (TE)
  Fusion method. Experiments show that higher frame rate inputs and TE Fusion
  yield improvements in motion understanding, yet there is still substantial room
  for enhancement. Our benchmark aims to guide and motivate the development of
  more capable video understanding models, emphasizing the importance of
  fine-grained motion comprehension. Project page: https://motion-bench.github.io .
paper_ai_keywords: vision language models (VLMs) fine-grained motion comprehension MotionBench motion-oriented question types Through-Encoder (TE) Fusion video feature compression