title: VidText: Towards Comprehensive Evaluation for Video Text Understanding
summary: Visual texts embedded in videos carry rich semantic information, which is
  crucial for both holistic video understanding and fine-grained reasoning about
  local human actions. However, existing video understanding benchmarks largely
  overlook textual information, while OCR-specific benchmarks are constrained to
  static images, limiting their ability to capture the interaction between text
  and dynamic visual contexts. To address this gap, we propose VidText, a new
  benchmark designed for comprehensive and in-depth evaluation of video text
  understanding. VidText offers the following key features: 1) It covers a wide
  range of real-world scenarios and supports multilingual content, encompassing
  diverse settings where video text naturally appears. 2) It introduces a
  hierarchical evaluation framework with video-level, clip-level, and
  instance-level tasks, enabling assessment of both global summarization and
  local retrieval capabilities. 3) The benchmark also introduces a set of paired
  perception reasoning tasks, ranging from visual text perception to cross-modal
  reasoning between textual and visual information. Extensive experiments on 18
  state-of-the-art Large Multimodal Models (LMMs) reveal that current models
  struggle across most tasks, with significant room for improvement. Further
  analysis highlights the impact of both model-intrinsic factors, such as input
  resolution and OCR capability, and external factors, including the use of
  auxiliary information and Chain-of-Thought reasoning strategies. We hope
  VidText will fill the current gap in video understanding benchmarks and serve
  as a foundation for future research on multimodal reasoning with video text in
  dynamic environments.
paper_authors._id: 68395c867f983113faf29005
paper_authors.name: Zhoufaran Yang
paper_authors._id: 68395c867f983113faf29006
paper_authors.name: Yan Shu
paper_authors._id: 68395c867f983113faf29007
paper_authors.name: Zhifei Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-02T07:46:17.645Z
paper_authors._id: 6464c4ef92773d5eeb588525
paper_authors.avatarUrl: /avatars/65fc839453d892016640249cc1acf277.svg
paper_authors.fullname: Zhifei Yang
paper_authors.type: user
paper_authors.user: yangzhifei
paper_authors._id: 68395c867f983113faf29008
paper_authors.name: Yan Zhang
paper_authors._id: 68395c867f983113faf29009
paper_authors.name: Yu Li
paper_authors._id: 68395c867f983113faf2900a
paper_authors.name: Keyang Lu
paper_authors._id: 68395c867f983113faf2900b
paper_authors.name: Gangyan Zeng
paper_authors._id: 68395c867f983113faf2900c
paper_authors.name: Shaohui Liu
paper_authors._id: 68395c867f983113faf2900d
paper_authors.name: Yu Zhou
paper_authors._id: 68395c867f983113faf2900e
paper_authors.name: Nicu Sebe
paper_title: VidText: Towards Comprehensive Evaluation for Video Text Understanding
paper_summary: Visual texts embedded in videos carry rich semantic information, which is
  crucial for both holistic video understanding and fine-grained reasoning about
  local human actions. However, existing video understanding benchmarks largely
  overlook textual information, while OCR-specific benchmarks are constrained to
  static images, limiting their ability to capture the interaction between text
  and dynamic visual contexts. To address this gap, we propose VidText, a new
  benchmark designed for comprehensive and in-depth evaluation of video text
  understanding. VidText offers the following key features: 1) It covers a wide
  range of real-world scenarios and supports multilingual content, encompassing
  diverse settings where video text naturally appears. 2) It introduces a
  hierarchical evaluation framework with video-level, clip-level, and
  instance-level tasks, enabling assessment of both global summarization and
  local retrieval capabilities. 3) The benchmark also introduces a set of paired
  perception reasoning tasks, ranging from visual text perception to cross-modal
  reasoning between textual and visual information. Extensive experiments on 18
  state-of-the-art Large Multimodal Models (LMMs) reveal that current models
  struggle across most tasks, with significant room for improvement. Further
  analysis highlights the impact of both model-intrinsic factors, such as input
  resolution and OCR capability, and external factors, including the use of
  auxiliary information and Chain-of-Thought reasoning strategies. We hope
  VidText will fill the current gap in video understanding benchmarks and serve
  as a foundation for future research on multimodal reasoning with video text in
  dynamic environments.
paper_ai_summary: VidText is a new benchmark that evaluates video text understanding across various tasks, covering global summarization and local retrieval, and highlights challenges for current multimodal models.
paper_ai_keywords: LMMs video text perception cross-modal reasoning Chain-of-Thought reasoning