title: CSVQA: A Chinese Multimodal Benchmark for Evaluating STEM Reasoning
    Capabilities of VLMs
summary: Vision-Language Models (VLMs) have demonstrated remarkable progress in
  multimodal understanding, yet their capabilities for scientific reasoning
  remains inadequately assessed. Current multimodal benchmarks predominantly
  evaluate generic image comprehension or text-driven reasoning, lacking
  authentic scientific contexts that require domain-specific knowledge
  integration with visual evidence analysis. To fill this gap, we present CSVQA,
  a diagnostic multimodal benchmark specifically designed for evaluating
  scientific reasoning through domain-grounded visual question answering.Our
  benchmark features 1,378 carefully constructed question-answer pairs spanning
  diverse STEM disciplines, each demanding domain knowledge, integration of
  visual evidence, and higher-order reasoning. Compared to prior multimodal
  benchmarks, CSVQA places greater emphasis on real-world scientific content and
  complex reasoning.We additionally propose a rigorous evaluation protocol to
  systematically assess whether model predictions are substantiated by valid
  intermediate reasoning steps based on curated explanations. Our comprehensive
  evaluation of 15 VLMs on this benchmark reveals notable performance
  disparities, as even the top-ranked proprietary model attains only 49.6\%
  accuracy.This empirical evidence underscores the pressing need for advancing
  scientific reasoning capabilities in VLMs. Our CSVQA is released at
  https://huggingface.co/datasets/Skywork/CSVQA.
paper_authors._id: 683fc08da33aeee1124887c4
paper_authors.name: Ai Jian
paper_authors._id: 683fc08da33aeee1124887c5
paper_authors.name: Weijie Qiu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T09:56:39.772Z
paper_authors._id: 660aab2c878289c5b34f9e97
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/660aab2c878289c5b34f9e97/yxx1-lR8x5o6KaEpZDXQq.jpeg
paper_authors.fullname: weijie qiu
paper_authors.type: user
paper_authors.user: qiuwj
paper_authors._id: 683fc08da33aeee1124887c6
paper_authors.name: Xiaokun Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T08:55:01.766Z
paper_authors._id: 62be9b5aae56e75e4d689e7c
paper_authors.avatarUrl: /avatars/6772bc09d6eeb4e86b1210481be91720.svg
paper_authors.fullname: wangxiaokun
paper_authors.type: user
paper_authors.user: shawn0wang
paper_authors._id: 683fc08da33aeee1124887c7
paper_authors.name: Peiyu Wang
paper_authors._id: 683fc08da33aeee1124887c8
paper_authors.name: Yunzhuo Hao
paper_authors._id: 683fc08da33aeee1124887c9
paper_authors.name: Jiangbo Pei
paper_authors._id: 683fc08da33aeee1124887ca
paper_authors.name: Yichen Wei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-04T10:25:28.093Z
paper_authors._id: 66d3ff488da15c5151c372fb
paper_authors.avatarUrl: /avatars/4e3ed8b675c822e768e17def7604f0d9.svg
paper_authors.fullname: Yichen Wei
paper_authors.type: user
paper_authors.user: rockman24
paper_authors._id: 683fc08da33aeee1124887cb
paper_authors.name: Yi Peng
paper_authors._id: 683fc08da33aeee1124887cc
paper_authors.name: Xuchen Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-04T10:25:14.285Z
paper_authors._id: 6462b241b438438da3c25a5d
paper_authors.avatarUrl: /avatars/606a67f1be639c9a5e36f293abd5f27a.svg
paper_authors.fullname: Xuchen Song
paper_authors.type: user
paper_authors.user: xuchensong
paper_title: CSVQA: A Chinese Multimodal Benchmark for Evaluating STEM Reasoning
    Capabilities of VLMs
paper_summary: Vision-Language Models (VLMs) have demonstrated remarkable progress in
  multimodal understanding, yet their capabilities for scientific reasoning
  remains inadequately assessed. Current multimodal benchmarks predominantly
  evaluate generic image comprehension or text-driven reasoning, lacking
  authentic scientific contexts that require domain-specific knowledge
  integration with visual evidence analysis. To fill this gap, we present CSVQA,
  a diagnostic multimodal benchmark specifically designed for evaluating
  scientific reasoning through domain-grounded visual question answering.Our
  benchmark features 1,378 carefully constructed question-answer pairs spanning
  diverse STEM disciplines, each demanding domain knowledge, integration of
  visual evidence, and higher-order reasoning. Compared to prior multimodal
  benchmarks, CSVQA places greater emphasis on real-world scientific content and
  complex reasoning.We additionally propose a rigorous evaluation protocol to
  systematically assess whether model predictions are substantiated by valid
  intermediate reasoning steps based on curated explanations. Our comprehensive
  evaluation of 15 VLMs on this benchmark reveals notable performance
  disparities, as even the top-ranked proprietary model attains only 49.6\%
  accuracy.This empirical evidence underscores the pressing need for advancing
  scientific reasoning capabilities in VLMs. Our CSVQA is released at
  https://huggingface.co/datasets/Skywork/CSVQA.
paper_ai_summary: A new benchmark, CSVQA, evaluates scientific reasoning in vision-language models through domain-specific visual question answering, highlighting the need for improvement in these models.
paper_ai_keywords: Vision-Language Models multimodal benchmark scientific reasoning domain-grounded visual question answering domain-specific knowledge higher-order reasoning evaluation protocol intermediate reasoning steps curated explanations