title: Caption Anything in Video: Fine-grained Object-centric Captioning via
    Spatiotemporal Multimodal Prompting
summary: We present CAT-V (Caption AnyThing in Video), a training-free framework for
  fine-grained object-centric video captioning that enables detailed descriptions
  of user-selected objects through time. CAT-V integrates three key components: a
  Segmenter based on SAMURAI for precise object segmentation across frames, a
  Temporal Analyzer powered by TRACE-Uni for accurate event boundary detection
  and temporal analysis, and a Captioner using InternVL-2.5 for generating
  detailed object-centric descriptions. Through spatiotemporal visual prompts and
  chain-of-thought reasoning, our framework generates detailed, temporally-aware
  descriptions of objects' attributes, actions, statuses, interactions, and
  environmental contexts without requiring additional training data. CAT-V
  supports flexible user interactions through various visual prompts (points,
  bounding boxes, and irregular regions) and maintains temporal sensitivity by
  tracking object states and interactions across different time segments. Our
  approach addresses limitations of existing video captioning methods, which
  either produce overly abstract descriptions or lack object-level precision,
  enabling fine-grained, object-specific descriptions while maintaining temporal
  coherence and spatial accuracy. The GitHub repository for this project is
  available at https://github.com/yunlong10/CAT-V
paper_authors._id: 67f7cebba5c42b93f487a2b3
paper_authors.name: Yunlong Tang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-10T15:45:42.596Z
paper_authors._id: 6344c87f0f69ad8aa61dfcf6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6344c87f0f69ad8aa61dfcf6/m4QiQ_4c6GfOIZZFICy7T.jpeg
paper_authors.fullname: Yolo Y. Tang
paper_authors.type: user
paper_authors.user: yunlong10
paper_authors._id: 67f7cebba5c42b93f487a2b4
paper_authors.name: Jing Bi
paper_authors._id: 67f7cebba5c42b93f487a2b5
paper_authors.name: Chao Huang
paper_authors._id: 67f7cebba5c42b93f487a2b6
paper_authors.name: Susan Liang
paper_authors._id: 67f7cebba5c42b93f487a2b7
paper_authors.name: Daiki Shimada
paper_authors._id: 67f7cebba5c42b93f487a2b8
paper_authors.name: Hang Hua
paper_authors._id: 67f7cebba5c42b93f487a2b9
paper_authors.name: Yunzhong Xiao
paper_authors._id: 67f7cebba5c42b93f487a2ba
paper_authors.name: Yizhi Song
paper_authors._id: 67f7cebba5c42b93f487a2bb
paper_authors.name: Pinxin Liu
paper_authors._id: 67f7cebba5c42b93f487a2bc
paper_authors.name: Mingqian Feng
paper_authors._id: 67f7cebba5c42b93f487a2bd
paper_authors.name: Junjia Guo
paper_authors._id: 67f7cebba5c42b93f487a2be
paper_authors.name: Zhuo Liu
paper_authors._id: 67f7cebba5c42b93f487a2bf
paper_authors.name: Luchuan Song
paper_authors._id: 67f7cebba5c42b93f487a2c0
paper_authors.name: Ali Vosoughi
paper_authors._id: 67f7cebba5c42b93f487a2c1
paper_authors.name: Jinxi He
paper_authors._id: 67f7cebba5c42b93f487a2c2
paper_authors.name: Liu He
paper_authors._id: 67f7cebba5c42b93f487a2c3
paper_authors.name: Zeliang Zhang
paper_authors._id: 67f7cebba5c42b93f487a2c4
paper_authors.name: Jiebo Luo
paper_authors._id: 67f7cebba5c42b93f487a2c5
paper_authors.name: Chenliang Xu
paper_title: Caption Anything in Video: Fine-grained Object-centric Captioning via
    Spatiotemporal Multimodal Prompting
paper_summary: We present CAT-V (Caption AnyThing in Video), a training-free framework for
  fine-grained object-centric video captioning that enables detailed descriptions
  of user-selected objects through time. CAT-V integrates three key components: a
  Segmenter based on SAMURAI for precise object segmentation across frames, a
  Temporal Analyzer powered by TRACE-Uni for accurate event boundary detection
  and temporal analysis, and a Captioner using InternVL-2.5 for generating
  detailed object-centric descriptions. Through spatiotemporal visual prompts and
  chain-of-thought reasoning, our framework generates detailed, temporally-aware
  descriptions of objects' attributes, actions, statuses, interactions, and
  environmental contexts without requiring additional training data. CAT-V
  supports flexible user interactions through various visual prompts (points,
  bounding boxes, and irregular regions) and maintains temporal sensitivity by
  tracking object states and interactions across different time segments. Our
  approach addresses limitations of existing video captioning methods, which
  either produce overly abstract descriptions or lack object-level precision,
  enabling fine-grained, object-specific descriptions while maintaining temporal
  coherence and spatial accuracy. The GitHub repository for this project is
  available at https://github.com/yunlong10/CAT-V
paper_ai_summary: CAT-V provides detailed video object-centric captions using spatiotemporal prompts without additional training, integrating segmentation, temporal analysis, and caption generation.
paper_ai_keywords: SAMURAI TRACE-Uni InternVL-2.5 spatiotemporal visual prompts chain-of-thought reasoning object segmentation event boundary detection temporal analysis object-centric descriptions temporal sensitivity flexible user interactions temporal coherence spatial accuracy