CSVQA: A Chinese Multimodal Benchmark for Evaluating STEM Reasoning
  Capabilities of VLMs
Vision-Language Models (VLMs) have demonstrated remarkable progress in
multimodal understanding, yet their capabilities for scientific reasoning
remains inadequately assessed. Current multimodal benchmarks predominantly
evaluate generic image comprehension or text-driven reasoning, lacking
authentic scientific contexts that require domain-specific knowledge
integration with visual evidence analysis. To fill this gap, we present CSVQA,
a diagnostic multimodal benchmark specifically designed for evaluating
scientific reasoning through domain-grounded visual question answering.Our
benchmark features 1,378 carefully constructed question-answer pairs spanning
diverse STEM disciplines, each demanding domain knowledge, integration of
visual evidence, and higher-order reasoning. Compared to prior multimodal
benchmarks, CSVQA places greater emphasis on real-world scientific content and
complex reasoning.We additionally propose a rigorous evaluation protocol to
systematically assess whether model predictions are substantiated by valid
intermediate reasoning steps based on curated explanations. Our comprehensive
evaluation of 15 VLMs on this benchmark reveals notable performance
disparities, as even the top-ranked proprietary model attains only 49.6\%
accuracy.This empirical evidence underscores the pressing need for advancing
scientific reasoning capabilities in VLMs. Our CSVQA is released at
https://huggingface.co/datasets/Skywork/CSVQA.
683fc08da33aeee1124887c4
Ai Jian
683fc08da33aeee1124887c5
Weijie Qiu
claimed_verified
2025-06-04T09:56:39.772Z
660aab2c878289c5b34f9e97
https://cdn-avatars.huggingface.co/v1/production/uploads/660aab2c878289c5b34f9e97/yxx1-lR8x5o6KaEpZDXQq.jpeg
weijie qiu
user
qiuwj
683fc08da33aeee1124887c6
Xiaokun Wang
claimed_verified
2025-06-04T08:55:01.766Z
62be9b5aae56e75e4d689e7c
/avatars/6772bc09d6eeb4e86b1210481be91720.svg
wangxiaokun
user
shawn0wang
683fc08da33aeee1124887c7
Peiyu Wang
683fc08da33aeee1124887c8
Yunzhuo Hao
683fc08da33aeee1124887c9
Jiangbo Pei
683fc08da33aeee1124887ca
Yichen Wei
admin_assigned
2025-06-04T10:25:28.093Z
66d3ff488da15c5151c372fb
/avatars/4e3ed8b675c822e768e17def7604f0d9.svg
Yichen Wei
user
rockman24
683fc08da33aeee1124887cb
Yi Peng
683fc08da33aeee1124887cc
Xuchen Song
admin_assigned
2025-06-04T10:25:14.285Z
6462b241b438438da3c25a5d
/avatars/606a67f1be639c9a5e36f293abd5f27a.svg
Xuchen Song
user
xuchensong
CSVQA: A Chinese Multimodal Benchmark for Evaluating STEM Reasoning
  Capabilities of VLMs
Vision-Language Models (VLMs) have demonstrated remarkable progress in
multimodal understanding, yet their capabilities for scientific reasoning
remains inadequately assessed. Current multimodal benchmarks predominantly
evaluate generic image comprehension or text-driven reasoning, lacking
authentic scientific contexts that require domain-specific knowledge
integration with visual evidence analysis. To fill this gap, we present CSVQA,
a diagnostic multimodal benchmark specifically designed for evaluating
scientific reasoning through domain-grounded visual question answering.Our
benchmark features 1,378 carefully constructed question-answer pairs spanning
diverse STEM disciplines, each demanding domain knowledge, integration of
visual evidence, and higher-order reasoning. Compared to prior multimodal
benchmarks, CSVQA places greater emphasis on real-world scientific content and
complex reasoning.We additionally propose a rigorous evaluation protocol to
systematically assess whether model predictions are substantiated by valid
intermediate reasoning steps based on curated explanations. Our comprehensive
evaluation of 15 VLMs on this benchmark reveals notable performance
disparities, as even the top-ranked proprietary model attains only 49.6\%
accuracy.This empirical evidence underscores the pressing need for advancing
scientific reasoning capabilities in VLMs. Our CSVQA is released at
https://huggingface.co/datasets/Skywork/CSVQA.
A new benchmark, CSVQA, evaluates scientific reasoning in vision-language models through domain-specific visual question answering, highlighting the need for improvement in these models.
Vision-Language Models multimodal benchmark scientific reasoning domain-grounded visual question answering domain-specific knowledge higher-order reasoning evaluation protocol intermediate reasoning steps curated explanations