title: MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based
    Scientific Research
summary: Scientific research demands sophisticated reasoning over multimodal data, a
  challenge especially prevalent in biology. Despite recent advances in
  multimodal large language models (MLLMs) for AI-assisted research, existing
  multimodal reasoning benchmarks only target up to college-level difficulty,
  while research-level benchmarks emphasize lower-level perception, falling short
  of the complex multimodal reasoning needed for scientific discovery. To bridge
  this gap, we introduce MicroVQA, a visual-question answering (VQA) benchmark
  designed to assess three reasoning capabilities vital in research workflows:
  expert image understanding, hypothesis generation, and experiment proposal.
  MicroVQA consists of 1,042 multiple-choice questions (MCQs) curated by biology
  experts across diverse microscopy modalities, ensuring VQA samples represent
  real scientific practice. In constructing the benchmark, we find that standard
  MCQ generation methods induce language shortcuts, motivating a new two-stage
  pipeline: an optimized LLM prompt structures question-answer pairs into MCQs;
  then, an agent-based `RefineBot' updates them to remove shortcuts. Benchmarking
  on state-of-the-art MLLMs reveal a peak performance of 53\%; models with
  smaller LLMs only slightly underperform top models, suggesting that
  language-based reasoning is less challenging than multimodal reasoning; and
  tuning with scientific articles enhances performance. Expert analysis of
  chain-of-thought responses shows that perception errors are the most frequent,
  followed by knowledge errors and then overgeneralization errors. These insights
  highlight the challenges in multimodal scientific reasoning, showing MicroVQA
  is a valuable resource advancing AI-driven biomedical research. MicroVQA is
  available at https://huggingface.co/datasets/jmhb/microvqa, and project page at
  https://jmhb0.github.io/microvqa.
paper_authors._id: 67d8d99a0983992037cdf33f
paper_authors.name: James Burgess
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-18T08:08:18.287Z
paper_authors._id: 650871aeb44445e9b3625c7b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/650871aeb44445e9b3625c7b/mtx3EnkuNF4z29IosnhaQ.png
paper_authors.fullname: James Burgess
paper_authors.type: user
paper_authors.user: jmhb
paper_authors._id: 67d8d99a0983992037cdf340
paper_authors.name: Jeffrey J Nirschl
paper_authors._id: 67d8d99a0983992037cdf341
paper_authors.name: Laura Bravo-Sánchez
paper_authors._id: 67d8d99a0983992037cdf342
paper_authors.name: Alejandro Lozano
paper_authors._id: 67d8d99a0983992037cdf343
paper_authors.name: Sanket Rajan Gupte
paper_authors._id: 67d8d99a0983992037cdf344
paper_authors.name: Jesus G. Galaz-Montoya
paper_authors._id: 67d8d99a0983992037cdf345
paper_authors.name: Yuhui Zhang
paper_authors._id: 67d8d99a0983992037cdf346
paper_authors.name: Yuchang Su
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-18T09:27:03.753Z
paper_authors._id: 666aa5183263a8feca6b7003
paper_authors.avatarUrl: /avatars/6ac4d52e8abea0df9f83da408502c076.svg
paper_authors.fullname: Yuchang Su
paper_authors.type: user
paper_authors.user: suyc21
paper_authors._id: 67d8d99a0983992037cdf347
paper_authors.name: Disha Bhowmik
paper_authors._id: 67d8d99a0983992037cdf348
paper_authors.name: Zachary Coman
paper_authors._id: 67d8d99a0983992037cdf349
paper_authors.name: Sarina M. Hasan
paper_authors._id: 67d8d99a0983992037cdf34a
paper_authors.name: Alexandra Johannesson
paper_authors._id: 67d8d99a0983992037cdf34b
paper_authors.name: William D. Leineweber
paper_authors._id: 67d8d99a0983992037cdf34c
paper_authors.name: Malvika G Nair
paper_authors._id: 67d8d99a0983992037cdf34d
paper_authors.name: Ridhi Yarlagadda
paper_authors._id: 67d8d99a0983992037cdf34e
paper_authors.name: Connor Zuraski
paper_authors._id: 67d8d99a0983992037cdf34f
paper_authors.name: Wah Chiu
paper_authors._id: 67d8d99a0983992037cdf350
paper_authors.name: Sarah Cohen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-18T09:26:24.760Z
paper_authors._id: 655b8dbb83186f133f7f8a98
paper_authors.avatarUrl: /avatars/15f41d0efb3a59a2e389cdb5338e0c1e.svg
paper_authors.fullname: Sarah Cohen
paper_authors.type: user
paper_authors.user: shcohen
paper_authors._id: 67d8d99a0983992037cdf351
paper_authors.name: Jan N. Hansen
paper_authors._id: 67d8d99a0983992037cdf352
paper_authors.name: Manuel D Leonetti
paper_authors._id: 67d8d99a0983992037cdf353
paper_authors.name: Chad Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-18T09:25:42.249Z
paper_authors._id: 64fc5c1cc45dd732acc2ec48
paper_authors.avatarUrl: /avatars/b1f072cbfec014f1a054d4a433cff93c.svg
paper_authors.fullname: Chad Liu
paper_authors.type: user
paper_authors.user: chadliu
paper_authors._id: 67d8d99a0983992037cdf354
paper_authors.name: Emma Lundberg
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-18T09:25:33.946Z
paper_authors._id: 678af263320331c7e008f842
paper_authors.avatarUrl: /avatars/e2cde80f018f3dd278000270fdbc104d.svg
paper_authors.fullname: emma lundberg
paper_authors.type: user
paper_authors.user: lundbergemma
paper_authors._id: 67d8d99a0983992037cdf355
paper_authors.name: Serena Yeung-Levy
paper_title: MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based
    Scientific Research
paper_summary: Scientific research demands sophisticated reasoning over multimodal data, a
  challenge especially prevalent in biology. Despite recent advances in
  multimodal large language models (MLLMs) for AI-assisted research, existing
  multimodal reasoning benchmarks only target up to college-level difficulty,
  while research-level benchmarks emphasize lower-level perception, falling short
  of the complex multimodal reasoning needed for scientific discovery. To bridge
  this gap, we introduce MicroVQA, a visual-question answering (VQA) benchmark
  designed to assess three reasoning capabilities vital in research workflows:
  expert image understanding, hypothesis generation, and experiment proposal.
  MicroVQA consists of 1,042 multiple-choice questions (MCQs) curated by biology
  experts across diverse microscopy modalities, ensuring VQA samples represent
  real scientific practice. In constructing the benchmark, we find that standard
  MCQ generation methods induce language shortcuts, motivating a new two-stage
  pipeline: an optimized LLM prompt structures question-answer pairs into MCQs;
  then, an agent-based `RefineBot' updates them to remove shortcuts. Benchmarking
  on state-of-the-art MLLMs reveal a peak performance of 53\%; models with
  smaller LLMs only slightly underperform top models, suggesting that
  language-based reasoning is less challenging than multimodal reasoning; and
  tuning with scientific articles enhances performance. Expert analysis of
  chain-of-thought responses shows that perception errors are the most frequent,
  followed by knowledge errors and then overgeneralization errors. These insights
  highlight the challenges in multimodal scientific reasoning, showing MicroVQA
  is a valuable resource advancing AI-driven biomedical research. MicroVQA is
  available at https://huggingface.co/datasets/jmhb/microvqa, and project page at
  https://jmhb0.github.io/microvqa.
paper_ai_keywords: multimodal large language models visual-question answering (VQA) multiple-choice questions (MCQs) biology experts microscopy modalities standard MCQ generation methods optimized LLM prompt agent-based `RefineBot' chain-of-thought responses perception errors knowledge errors overgeneralization errors