VideoVista-CulturalLingo: 360^circ Horizons-Bridging Cultures,
  Languages, and Domains in Video Comprehension
Assessing the video comprehension capabilities of multimodal AI systems can
effectively measure their understanding and reasoning abilities. Most video
evaluation benchmarks are limited to a single language, typically English, and
predominantly feature videos rooted in Western cultural contexts. In this
paper, we present VideoVista-CulturalLingo, the first video evaluation
benchmark designed to bridge cultural, linguistic, and domain divide in video
comprehension. Our work differs from existing benchmarks in the following ways:
1) Cultural diversity, incorporating cultures from China, North America, and
Europe; 2) Multi-linguistics, with questions presented in Chinese and
English-two of the most widely spoken languages; and 3) Broad domain, featuring
videos sourced from hundreds of human-created domains. VideoVista-CulturalLingo
contains 1,389 videos and 3,134 QA pairs, and we have evaluated 24 recent
open-source or proprietary video large models. From the experiment results, we
observe that: 1) Existing models perform worse on Chinese-centric questions
than Western-centric ones, particularly those related to Chinese history; 2)
Current open-source models still exhibit limitations in temporal understanding,
especially in the Event Localization task, achieving a maximum score of only
45.2%; 3) Mainstream models demonstrate strong performance in general
scientific questions, while open-source models demonstrate weak performance in
mathematics.
680f56b8da9639d22c64443f
Xinyu Chen
claimed_verified
2025-04-28T12:50:03.092Z
64d9da538767727dff1e8f19
/avatars/aaad38795007c6cbcb94c7eed1706e51.svg
Yu
user
Ghaser
680f56b8da9639d22c644440
Yunxin Li
claimed_verified
2025-04-29T08:01:07.557Z
62fdb01bc1588e1d4c6c1a7c
/avatars/bd03085995b1c34e0ac8a845cf2c4e83.svg
Yunxin Li
user
YunxinLi
680f56b8da9639d22c644441
Haoyuan Shi
admin_assigned
2025-04-29T12:32:07.664Z
652fb8bcc9dd2692a25ef2e3
/avatars/461e6cc1c3441cde18192b080b0b8576.svg
Haoyuan Shi
user
MrSunshy
680f56b8da9639d22c644442
Baotian Hu
680f56b8da9639d22c644443
Wenhan Luo
admin_assigned
2025-04-29T12:32:20.853Z
65f865cf806814aec87d45ce
/avatars/6b931c1efe20422a5985fce6a9e36aed.svg
Wenhan Luo
user
whluo
680f56b8da9639d22c644444
Yaowei Wang
680f56b8da9639d22c644445
Min Zhang
VideoVista-CulturalLingo: 360^circ Horizons-Bridging Cultures,
  Languages, and Domains in Video Comprehension
Assessing the video comprehension capabilities of multimodal AI systems can
effectively measure their understanding and reasoning abilities. Most video
evaluation benchmarks are limited to a single language, typically English, and
predominantly feature videos rooted in Western cultural contexts. In this
paper, we present VideoVista-CulturalLingo, the first video evaluation
benchmark designed to bridge cultural, linguistic, and domain divide in video
comprehension. Our work differs from existing benchmarks in the following ways:
1) Cultural diversity, incorporating cultures from China, North America, and
Europe; 2) Multi-linguistics, with questions presented in Chinese and
English-two of the most widely spoken languages; and 3) Broad domain, featuring
videos sourced from hundreds of human-created domains. VideoVista-CulturalLingo
contains 1,389 videos and 3,134 QA pairs, and we have evaluated 24 recent
open-source or proprietary video large models. From the experiment results, we
observe that: 1) Existing models perform worse on Chinese-centric questions
than Western-centric ones, particularly those related to Chinese history; 2)
Current open-source models still exhibit limitations in temporal understanding,
especially in the Event Localization task, achieving a maximum score of only
45.2%; 3) Mainstream models demonstrate strong performance in general
scientific questions, while open-source models demonstrate weak performance in
mathematics.
A novel video comprehension benchmark evaluates multimodal AI systems across diverse cultures and languages, revealing performance gaps in specific domains and linguistic contexts.
video evaluation benchmark cultural diversity multi-linguistics Event Localization task video comprehension cultural context linguistic context domain diversity