title: Number it: Temporal Grounding Videos like Flipping Manga
summary: Video Large Language Models (Vid-LLMs) have made remarkable advancements in
  comprehending video content for QA dialogue. However, they struggle to extend
  this visual understanding to tasks requiring precise temporal localization,
  known as Video Temporal Grounding (VTG). To address this gap, we introduce
  Number-Prompt (NumPro), a novel method that empowers Vid-LLMs to bridge visual
  comprehension with temporal grounding by adding unique numerical identifiers to
  each video frame. Treating a video as a sequence of numbered frame images,
  NumPro transforms VTG into an intuitive process: flipping through manga panels
  in sequence. This allows Vid-LLMs to "read" event timelines, accurately linking
  visual content with corresponding temporal information. Our experiments
  demonstrate that NumPro significantly boosts VTG performance of top-tier
  Vid-LLMs without additional computational cost. Furthermore, fine-tuning on a
  NumPro-enhanced dataset defines a new state-of-the-art for VTG, surpassing
  previous top-performing methods by up to 6.9\% in mIoU for moment retrieval and
  8.5\% in mAP for highlight detection. The code will be available at
  https://github.com/yongliang-wu/NumPro.
paper_authors._id: 673ad833abddf84949fd2952
paper_authors.name: Yongliang Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-18T07:47:14.506Z
paper_authors._id: 66f6bc97980d52c75c300511
paper_authors.avatarUrl: /avatars/f7c23c4b09701580b533212ec9b6e306.svg
paper_authors.fullname: Yongliang
paper_authors.type: user
paper_authors.user: Liang0223
paper_authors._id: 673ad833abddf84949fd2953
paper_authors.name: Xinting Hu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-18T07:51:51.989Z
paper_authors._id: 652845524def261e0febbf5b
paper_authors.avatarUrl: /avatars/92743e595cbce4e37a8fe7400b5051ba.svg
paper_authors.fullname: Xinting Hu
paper_authors.type: user
paper_authors.user: kaleidudu
paper_authors._id: 673ad833abddf84949fd2954
paper_authors.name: Yuyang Sun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-18T07:51:58.153Z
paper_authors._id: 655540892da2e4f12e6c743c
paper_authors.avatarUrl: /avatars/790a80f7f9c21e3bf09addb323a6467a.svg
paper_authors.fullname: Yuyang Sun
paper_authors.type: user
paper_authors.user: tms28k
paper_authors._id: 673ad833abddf84949fd2955
paper_authors.name: Yizhou Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-18T07:52:13.463Z
paper_authors._id: 667ea7f396704b4d8f95ba58
paper_authors.avatarUrl: /avatars/a3ec12d590e99cad07d3ecb618e5aacf.svg
paper_authors.fullname: Zhouyi Zhou
paper_authors.type: user
paper_authors.user: zhouzhouyi
paper_authors._id: 673ad833abddf84949fd2956
paper_authors.name: Wenbo Zhu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-18T07:47:04.587Z
paper_authors._id: 6434e89e7b82474801105f06
paper_authors.avatarUrl: /avatars/25d23d564d35e2ecb5ed6b1bbd0aa489.svg
paper_authors.fullname: Vito Zhu
paper_authors.type: user
paper_authors.user: Vito328
paper_authors._id: 673ad833abddf84949fd2957
paper_authors.name: Fengyun Rao
paper_authors._id: 673ad833abddf84949fd2958
paper_authors.name: Bernt Schiele
paper_authors._id: 673ad833abddf84949fd2959
paper_authors.name: Xu Yang
paper_title: Number it: Temporal Grounding Videos like Flipping Manga
paper_summary: Video Large Language Models (Vid-LLMs) have made remarkable advancements in
  comprehending video content for QA dialogue. However, they struggle to extend
  this visual understanding to tasks requiring precise temporal localization,
  known as Video Temporal Grounding (VTG). To address this gap, we introduce
  Number-Prompt (NumPro), a novel method that empowers Vid-LLMs to bridge visual
  comprehension with temporal grounding by adding unique numerical identifiers to
  each video frame. Treating a video as a sequence of numbered frame images,
  NumPro transforms VTG into an intuitive process: flipping through manga panels
  in sequence. This allows Vid-LLMs to "read" event timelines, accurately linking
  visual content with corresponding temporal information. Our experiments
  demonstrate that NumPro significantly boosts VTG performance of top-tier
  Vid-LLMs without additional computational cost. Furthermore, fine-tuning on a
  NumPro-enhanced dataset defines a new state-of-the-art for VTG, surpassing
  previous top-performing methods by up to 6.9\% in mIoU for moment retrieval and
  8.5\% in mAP for highlight detection. The code will be available at
  https://github.com/yongliang-wu/NumPro.
paper_ai_summary: Number-Prompt (NumPro) enhances Video Large Language Models (Vid-LLMs) for Video Temporal Grounding (VTG) by adding numerical identifiers to video frames, improving performance significantly without increased computational costs.
paper_ai_keywords: Vid-LLMs Video Temporal Grounding VTG Number-Prompt NumPro moment retrieval highlight detection