MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based
  Scientific Research
Scientific research demands sophisticated reasoning over multimodal data, a
challenge especially prevalent in biology. Despite recent advances in
multimodal large language models (MLLMs) for AI-assisted research, existing
multimodal reasoning benchmarks only target up to college-level difficulty,
while research-level benchmarks emphasize lower-level perception, falling short
of the complex multimodal reasoning needed for scientific discovery. To bridge
this gap, we introduce MicroVQA, a visual-question answering (VQA) benchmark
designed to assess three reasoning capabilities vital in research workflows:
expert image understanding, hypothesis generation, and experiment proposal.
MicroVQA consists of 1,042 multiple-choice questions (MCQs) curated by biology
experts across diverse microscopy modalities, ensuring VQA samples represent
real scientific practice. In constructing the benchmark, we find that standard
MCQ generation methods induce language shortcuts, motivating a new two-stage
pipeline: an optimized LLM prompt structures question-answer pairs into MCQs;
then, an agent-based `RefineBot' updates them to remove shortcuts. Benchmarking
on state-of-the-art MLLMs reveal a peak performance of 53\%; models with
smaller LLMs only slightly underperform top models, suggesting that
language-based reasoning is less challenging than multimodal reasoning; and
tuning with scientific articles enhances performance. Expert analysis of
chain-of-thought responses shows that perception errors are the most frequent,
followed by knowledge errors and then overgeneralization errors. These insights
highlight the challenges in multimodal scientific reasoning, showing MicroVQA
is a valuable resource advancing AI-driven biomedical research. MicroVQA is
available at https://huggingface.co/datasets/jmhb/microvqa, and project page at
https://jmhb0.github.io/microvqa.
67d8d99a0983992037cdf33f
James Burgess
claimed_verified
2025-03-18T08:08:18.287Z
650871aeb44445e9b3625c7b
https://cdn-avatars.huggingface.co/v1/production/uploads/650871aeb44445e9b3625c7b/mtx3EnkuNF4z29IosnhaQ.png
James Burgess
user
jmhb
67d8d99a0983992037cdf340
Jeffrey J Nirschl
67d8d99a0983992037cdf341
Laura Bravo-Sánchez
67d8d99a0983992037cdf342
Alejandro Lozano
67d8d99a0983992037cdf343
Sanket Rajan Gupte
67d8d99a0983992037cdf344
Jesus G. Galaz-Montoya
67d8d99a0983992037cdf345
Yuhui Zhang
claimed_verified
2025-03-21T11:46:28.151Z
62da55164398e21bf7f0e292
https://cdn-avatars.huggingface.co/v1/production/uploads/62da55164398e21bf7f0e292/xjKkG8IA2IZZqCdjApSh3.jpeg
Yuhui Zhang
user
yuhuizhang
67d8d99a0983992037cdf346
Yuchang Su
admin_assigned
2025-03-18T09:27:03.753Z
666aa5183263a8feca6b7003
/avatars/6ac4d52e8abea0df9f83da408502c076.svg
Yuchang Su
user
suyc21
67d8d99a0983992037cdf347
Disha Bhowmik
67d8d99a0983992037cdf348
Zachary Coman
67d8d99a0983992037cdf349
Sarina M. Hasan
67d8d99a0983992037cdf34a
Alexandra Johannesson
67d8d99a0983992037cdf34b
William D. Leineweber
67d8d99a0983992037cdf34c
Malvika G Nair
67d8d99a0983992037cdf34d
Ridhi Yarlagadda
67d8d99a0983992037cdf34e
Connor Zuraski
67d8d99a0983992037cdf34f
Wah Chiu
67d8d99a0983992037cdf350
Sarah Cohen
admin_assigned
2025-03-18T09:26:24.760Z
655b8dbb83186f133f7f8a98
/avatars/15f41d0efb3a59a2e389cdb5338e0c1e.svg
Sarah Cohen
user
shcohen
67d8d99a0983992037cdf351
Jan N. Hansen
67d8d99a0983992037cdf352
Manuel D Leonetti
67d8d99a0983992037cdf353
Chad Liu
admin_assigned
2025-03-18T09:25:42.249Z
64fc5c1cc45dd732acc2ec48
/avatars/b1f072cbfec014f1a054d4a433cff93c.svg
Chad Liu
user
chadliu
67d8d99a0983992037cdf354
Emma Lundberg
admin_assigned
2025-03-18T09:25:33.946Z
678af263320331c7e008f842
/avatars/e2cde80f018f3dd278000270fdbc104d.svg
emma lundberg
user
lundbergemma
67d8d99a0983992037cdf355
Serena Yeung-Levy
MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based
  Scientific Research
Scientific research demands sophisticated reasoning over multimodal data, a
challenge especially prevalent in biology. Despite recent advances in
multimodal large language models (MLLMs) for AI-assisted research, existing
multimodal reasoning benchmarks only target up to college-level difficulty,
while research-level benchmarks emphasize lower-level perception, falling short
of the complex multimodal reasoning needed for scientific discovery. To bridge
this gap, we introduce MicroVQA, a visual-question answering (VQA) benchmark
designed to assess three reasoning capabilities vital in research workflows:
expert image understanding, hypothesis generation, and experiment proposal.
MicroVQA consists of 1,042 multiple-choice questions (MCQs) curated by biology
experts across diverse microscopy modalities, ensuring VQA samples represent
real scientific practice. In constructing the benchmark, we find that standard
MCQ generation methods induce language shortcuts, motivating a new two-stage
pipeline: an optimized LLM prompt structures question-answer pairs into MCQs;
then, an agent-based `RefineBot' updates them to remove shortcuts. Benchmarking
on state-of-the-art MLLMs reveal a peak performance of 53\%; models with
smaller LLMs only slightly underperform top models, suggesting that
language-based reasoning is less challenging than multimodal reasoning; and
tuning with scientific articles enhances performance. Expert analysis of
chain-of-thought responses shows that perception errors are the most frequent,
followed by knowledge errors and then overgeneralization errors. These insights
highlight the challenges in multimodal scientific reasoning, showing MicroVQA
is a valuable resource advancing AI-driven biomedical research. MicroVQA is
available at https://huggingface.co/datasets/jmhb/microvqa, and project page at
https://jmhb0.github.io/microvqa.
MicroVQA is a visual-question answering benchmark for scientific multimodal reasoning, assessing expert image understanding, hypothesis generation, and experiment proposal.
multimodal large language models visual-question answering expert image understanding hypothesis generation experiment proposal multiple-choice questions LLM prompt RefineBot language-based reasoning multimodal reasoning chain-of-thought responses perception errors knowledge errors overgeneralization errors