title: μ-Bench: A Vision-Language Benchmark for Microscopy Understanding
summary: Recent advances in microscopy have enabled the rapid generation of terabytes
  of image data in cell biology and biomedical research. Vision-language models
  (VLMs) offer a promising solution for large-scale biological image analysis,
  enhancing researchers' efficiency, identifying new image biomarkers, and
  accelerating hypothesis generation and scientific discovery. However, there is
  a lack of standardized, diverse, and large-scale vision-language benchmarks to
  evaluate VLMs' perception and cognition capabilities in biological image
  understanding. To address this gap, we introduce {\mu}-Bench, an expert-curated
  benchmark encompassing 22 biomedical tasks across various scientific
  disciplines (biology, pathology), microscopy modalities (electron,
  fluorescence, light), scales (subcellular, cellular, tissue), and organisms in
  both normal and abnormal states. We evaluate state-of-the-art biomedical,
  pathology, and general VLMs on {\mu}-Bench and find that: i) current models
  struggle on all categories, even for basic tasks such as distinguishing
  microscopy modalities; ii) current specialist models fine-tuned on biomedical
  data often perform worse than generalist models; iii) fine-tuning in specific
  microscopy domains can cause catastrophic forgetting, eroding prior biomedical
  knowledge encoded in their base model. iv) weight interpolation between
  fine-tuned and pre-trained models offers one solution to forgetting and
  improves general performance across biomedical tasks. We release {\mu}-Bench
  under a permissive license to accelerate the research and development of
  microscopy foundation models.
paper_authors._id: 6685bfc85a5cdc0baee289cf
paper_authors.name: Alejandro Lozano
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-04T08:03:56.249Z
paper_authors._id: 630e56d6f6f6d700f50be105
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/630e56d6f6f6d700f50be105/_gzPXZeL2LP8oalMCriNN.jpeg
paper_authors.fullname: Alejandro Lozano
paper_authors.type: user
paper_authors.user: Alejandro98
paper_authors._id: 6685bfc85a5cdc0baee289d0
paper_authors.name: Jeffrey Nirschl
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-07-03T22:18:57.523Z
paper_authors._id: 6500e67e13f1546526bd373a
paper_authors.avatarUrl: /avatars/80ef1ce58e93b773dcbcc2d5b53175c2.svg
paper_authors.fullname: Jeff Nirschl
paper_authors.type: user
paper_authors.user: jnirschl
paper_authors._id: 6685bfc85a5cdc0baee289d1
paper_authors.name: James Burgess
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-07-03T21:23:29.887Z
paper_authors._id: 650871aeb44445e9b3625c7b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/650871aeb44445e9b3625c7b/mtx3EnkuNF4z29IosnhaQ.png
paper_authors.fullname: James Burgess
paper_authors.type: user
paper_authors.user: jmhb
paper_authors._id: 6685bfc85a5cdc0baee289d2
paper_authors.name: Sanket Rajan Gupte
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-07-03T22:21:03.466Z
paper_authors._id: 64e80e69ad1bac272b704647
paper_authors.avatarUrl: /avatars/540c254d9945efba4ec428241a729a38.svg
paper_authors.fullname: Sanket Gupte
paper_authors.type: user
paper_authors.user: sanketg
paper_authors._id: 6685bfc85a5cdc0baee289d3
paper_authors.name: Yuhui Zhang
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-07-03T22:01:50.861Z
paper_authors._id: 62da55164398e21bf7f0e292
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62da55164398e21bf7f0e292/xjKkG8IA2IZZqCdjApSh3.jpeg
paper_authors.fullname: Yuhui Zhang
paper_authors.type: user
paper_authors.user: yuhuizhang
paper_authors._id: 6685bfc85a5cdc0baee289d4
paper_authors.name: Alyssa Unell
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-07-03T21:17:01.667Z
paper_authors._id: 6623f8d689e599f5169cb56f
paper_authors.avatarUrl: /avatars/24003f9ce9a1a7e34f24bf5b0fc2965d.svg
paper_authors.fullname: Alyssa Unell
paper_authors.type: user
paper_authors.user: aunell
paper_authors._id: 6685bfc85a5cdc0baee289d5
paper_authors.name: Serena Yeung-Levy
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-01-07T02:05:14.400Z
paper_authors._id: 677c8b2e92550a07fcad0f50
paper_authors.avatarUrl: /avatars/2be26e8f25e98cfe5b1d227ee0409cd0.svg
paper_authors.fullname: Serena Yeung-Levy
paper_authors.type: user
paper_authors.user: yeunglevy
paper_title: μ-Bench: A Vision-Language Benchmark for Microscopy Understanding
paper_summary: Recent advances in microscopy have enabled the rapid generation of terabytes
  of image data in cell biology and biomedical research. Vision-language models
  (VLMs) offer a promising solution for large-scale biological image analysis,
  enhancing researchers' efficiency, identifying new image biomarkers, and
  accelerating hypothesis generation and scientific discovery. However, there is
  a lack of standardized, diverse, and large-scale vision-language benchmarks to
  evaluate VLMs' perception and cognition capabilities in biological image
  understanding. To address this gap, we introduce {\mu}-Bench, an expert-curated
  benchmark encompassing 22 biomedical tasks across various scientific
  disciplines (biology, pathology), microscopy modalities (electron,
  fluorescence, light), scales (subcellular, cellular, tissue), and organisms in
  both normal and abnormal states. We evaluate state-of-the-art biomedical,
  pathology, and general VLMs on {\mu}-Bench and find that: i) current models
  struggle on all categories, even for basic tasks such as distinguishing
  microscopy modalities; ii) current specialist models fine-tuned on biomedical
  data often perform worse than generalist models; iii) fine-tuning in specific
  microscopy domains can cause catastrophic forgetting, eroding prior biomedical
  knowledge encoded in their base model. iv) weight interpolation between
  fine-tuned and pre-trained models offers one solution to forgetting and
  improves general performance across biomedical tasks. We release {\mu}-Bench
  under a permissive license to accelerate the research and development of
  microscopy foundation models.
paper_ai_keywords: Vision-language models (VLMs) biological image analysis vision-language benchmarks microscopy modalities subcellular cellular tissue {\mu}-Bench specialist models generalist models catastrophic forgetting weight interpolation microscopy foundation models