title: VideoVista-CulturalLingo: 360^circ Horizons-Bridging Cultures,
    Languages, and Domains in Video Comprehension
summary: Assessing the video comprehension capabilities of multimodal AI systems can
  effectively measure their understanding and reasoning abilities. Most video
  evaluation benchmarks are limited to a single language, typically English, and
  predominantly feature videos rooted in Western cultural contexts. In this
  paper, we present VideoVista-CulturalLingo, the first video evaluation
  benchmark designed to bridge cultural, linguistic, and domain divide in video
  comprehension. Our work differs from existing benchmarks in the following ways:
  1) Cultural diversity, incorporating cultures from China, North America, and
  Europe; 2) Multi-linguistics, with questions presented in Chinese and
  English-two of the most widely spoken languages; and 3) Broad domain, featuring
  videos sourced from hundreds of human-created domains. VideoVista-CulturalLingo
  contains 1,389 videos and 3,134 QA pairs, and we have evaluated 24 recent
  open-source or proprietary video large models. From the experiment results, we
  observe that: 1) Existing models perform worse on Chinese-centric questions
  than Western-centric ones, particularly those related to Chinese history; 2)
  Current open-source models still exhibit limitations in temporal understanding,
  especially in the Event Localization task, achieving a maximum score of only
  45.2%; 3) Mainstream models demonstrate strong performance in general
  scientific questions, while open-source models demonstrate weak performance in
  mathematics.
paper_authors._id: 680f56b8da9639d22c64443f
paper_authors.name: Xinyu Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-28T12:50:03.092Z
paper_authors._id: 64d9da538767727dff1e8f19
paper_authors.avatarUrl: /avatars/aaad38795007c6cbcb94c7eed1706e51.svg
paper_authors.fullname: Yu
paper_authors.type: user
paper_authors.user: Ghaser
paper_authors._id: 680f56b8da9639d22c644440
paper_authors.name: Yunxin Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-29T08:01:07.557Z
paper_authors._id: 62fdb01bc1588e1d4c6c1a7c
paper_authors.avatarUrl: /avatars/bd03085995b1c34e0ac8a845cf2c4e83.svg
paper_authors.fullname: Yunxin Li
paper_authors.type: user
paper_authors.user: YunxinLi
paper_authors._id: 680f56b8da9639d22c644441
paper_authors.name: Haoyuan Shi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-29T12:32:07.664Z
paper_authors._id: 652fb8bcc9dd2692a25ef2e3
paper_authors.avatarUrl: /avatars/461e6cc1c3441cde18192b080b0b8576.svg
paper_authors.fullname: Haoyuan Shi
paper_authors.type: user
paper_authors.user: MrSunshy
paper_authors._id: 680f56b8da9639d22c644442
paper_authors.name: Baotian Hu
paper_authors._id: 680f56b8da9639d22c644443
paper_authors.name: Wenhan Luo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-29T12:32:20.853Z
paper_authors._id: 65f865cf806814aec87d45ce
paper_authors.avatarUrl: /avatars/6b931c1efe20422a5985fce6a9e36aed.svg
paper_authors.fullname: Wenhan Luo
paper_authors.type: user
paper_authors.user: whluo
paper_authors._id: 680f56b8da9639d22c644444
paper_authors.name: Yaowei Wang
paper_authors._id: 680f56b8da9639d22c644445
paper_authors.name: Min Zhang
paper_title: VideoVista-CulturalLingo: 360^circ Horizons-Bridging Cultures,
    Languages, and Domains in Video Comprehension
paper_summary: Assessing the video comprehension capabilities of multimodal AI systems can
  effectively measure their understanding and reasoning abilities. Most video
  evaluation benchmarks are limited to a single language, typically English, and
  predominantly feature videos rooted in Western cultural contexts. In this
  paper, we present VideoVista-CulturalLingo, the first video evaluation
  benchmark designed to bridge cultural, linguistic, and domain divide in video
  comprehension. Our work differs from existing benchmarks in the following ways:
  1) Cultural diversity, incorporating cultures from China, North America, and
  Europe; 2) Multi-linguistics, with questions presented in Chinese and
  English-two of the most widely spoken languages; and 3) Broad domain, featuring
  videos sourced from hundreds of human-created domains. VideoVista-CulturalLingo
  contains 1,389 videos and 3,134 QA pairs, and we have evaluated 24 recent
  open-source or proprietary video large models. From the experiment results, we
  observe that: 1) Existing models perform worse on Chinese-centric questions
  than Western-centric ones, particularly those related to Chinese history; 2)
  Current open-source models still exhibit limitations in temporal understanding,
  especially in the Event Localization task, achieving a maximum score of only
  45.2%; 3) Mainstream models demonstrate strong performance in general
  scientific questions, while open-source models demonstrate weak performance in
  mathematics.
paper_ai_summary: A novel video comprehension benchmark evaluates multimodal AI systems across diverse cultures and languages, revealing performance gaps in specific domains and linguistic contexts.
paper_ai_keywords: video evaluation benchmark cultural diversity multi-linguistics Event Localization task video comprehension cultural context linguistic context domain diversity