title: BIOMEDICA: An Open Biomedical Image-Caption Archive, Dataset, and
    Vision-Language Models Derived from Scientific Literature
summary: The development of vision-language models (VLMs) is driven by large-scale and
  diverse multimodal datasets. However, progress toward generalist biomedical
  VLMs is limited by the lack of annotated, publicly accessible datasets across
  biology and medicine. Existing efforts are restricted to narrow domains,
  missing the full diversity of biomedical knowledge encoded in scientific
  literature. To address this gap, we introduce BIOMEDICA, a scalable,
  open-source framework to extract, annotate, and serialize the entirety of the
  PubMed Central Open Access subset into an easy-to-use, publicly accessible
  dataset.Our framework produces a comprehensive archive with over 24 million
  unique image-text pairs from over 6 million articles. Metadata and
  expert-guided annotations are also provided. We demonstrate the utility and
  accessibility of our resource by releasing BMCA-CLIP, a suite of CLIP-style
  models continuously pre-trained on the BIOMEDICA dataset via streaming,
  eliminating the need to download 27 TB of data locally.On average, our models
  achieve state-of-the-art performance across 40 tasks - spanning pathology,
  radiology, ophthalmology, dermatology, surgery, molecular biology,
  parasitology, and cell biology - excelling in zero-shot classification with a
  6.56% average improvement (as high as 29.8% and 17.5% in dermatology and
  ophthalmology, respectively), and stronger image-text retrieval, all while
  using 10x less compute. To foster reproducibility and collaboration, we release
  our codebase and dataset for the broader research community.
paper_authors._id: 678603c0c054524aa22be7e1
paper_authors.name: Alejandro Lozano
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-01-14T06:27:14.763Z
paper_authors._id: 643c3fb526f177a3e4181e05
paper_authors.avatarUrl: /avatars/4f08ca15c5fbd2f8bb3d5c24f3fcc7b8.svg
paper_authors.fullname: Alejandro Lozano
paper_authors.type: user
paper_authors.user: lozanoe
paper_authors._id: 678603c0c054524aa22be7e2
paper_authors.name: Min Woo Sun
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-01-14T06:45:53.756Z
paper_authors._id: 65ac61120844d9e0d67a9f89
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65ac61120844d9e0d67a9f89/OzA_yxFoeuyhmc3xdYYdP.jpeg
paper_authors.fullname: Min Woo Sun
paper_authors.type: user
paper_authors.user: minwoosun
paper_authors._id: 678603c0c054524aa22be7e3
paper_authors.name: James Burgess
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-15T08:49:11.461Z
paper_authors._id: 650871aeb44445e9b3625c7b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/650871aeb44445e9b3625c7b/mtx3EnkuNF4z29IosnhaQ.png
paper_authors.fullname: James Burgess
paper_authors.type: user
paper_authors.user: jmhb
paper_authors._id: 678603c0c054524aa22be7e4
paper_authors.name: Liangyu Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-15T08:49:09.051Z
paper_authors._id: 62b67da0f56de4396ca9e44b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1658586059273-62b67da0f56de4396ca9e44b.jpeg
paper_authors.fullname: Liangyu Chen
paper_authors.type: user
paper_authors.user: liangyuch
paper_authors._id: 678603c0c054524aa22be7e5
paper_authors.name: Jeffrey J Nirschl
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-01-14T06:27:14.763Z
paper_authors._id: 6500e67e13f1546526bd373a
paper_authors.avatarUrl: /avatars/80ef1ce58e93b773dcbcc2d5b53175c2.svg
paper_authors.fullname: Jeff Nirschl
paper_authors.type: user
paper_authors.user: jnirschl
paper_authors._id: 678603c0c054524aa22be7e6
paper_authors.name: Jeffrey Gu
paper_authors._id: 678603c0c054524aa22be7e7
paper_authors.name: Ivan Lopez
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-15T08:49:06.626Z
paper_authors._id: 653fe33e73325383c2c9f9ba
paper_authors.avatarUrl: /avatars/9f8863fbbc613915c70e620b7a147e4d.svg
paper_authors.fullname: Ivan Lopez
paper_authors.type: user
paper_authors.user: ivlopez
paper_authors._id: 678603c0c054524aa22be7e8
paper_authors.name: Josiah Aklilu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-16T21:55:50.124Z
paper_authors._id: 66844ab799dbd7c30a19a0fe
paper_authors.avatarUrl: /avatars/5a5f8fa530e127105812bd1d0e515db8.svg
paper_authors.fullname: Josiah Aklilu
paper_authors.type: user
paper_authors.user: josaklil-ai
paper_authors._id: 678603c0c054524aa22be7e9
paper_authors.name: Austin Wolfgang Katzer
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-01-15T00:04:26.115Z
paper_authors._id: 6786fb7ade13ae3961c03d4d
paper_authors.avatarUrl: /avatars/156658b3c24283a564472001dec63e5a.svg
paper_authors.fullname: Austin Katzer
paper_authors.type: user
paper_authors.user: Awkatzer
paper_authors._id: 678603c0c054524aa22be7ea
paper_authors.name: Collin Chiu
paper_authors._id: 678603c0c054524aa22be7eb
paper_authors.name: Anita Rau
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-18T09:35:17.795Z
paper_authors._id: 6733c7f1d3521b36243b3c91
paper_authors.avatarUrl: /avatars/a070c5296fbce770822b1897157ddbe3.svg
paper_authors.fullname: Anita
paper_authors.type: user
paper_authors.user: anitar03
paper_authors._id: 678603c0c054524aa22be7ec
paper_authors.name: Xiaohan Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-16T08:31:48.670Z
paper_authors._id: 65703fab7f50602340d23704
paper_authors.avatarUrl: /avatars/324c45f5fba9cd8c38a89b30427c06b4.svg
paper_authors.fullname: Xiaohan Wang
paper_authors.type: user
paper_authors.user: nicholswang
paper_authors._id: 678603c0c054524aa22be7ed
paper_authors.name: Yuhui Zhang
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-01-14T06:28:22.730Z
paper_authors._id: 62da55164398e21bf7f0e292
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62da55164398e21bf7f0e292/xjKkG8IA2IZZqCdjApSh3.jpeg
paper_authors.fullname: Yuhui Zhang
paper_authors.type: user
paper_authors.user: yuhuizhang
paper_authors._id: 678603c0c054524aa22be7ee
paper_authors.name: Alfred Seunghoon Song
paper_authors._id: 678603c0c054524aa22be7ef
paper_authors.name: Robert Tibshirani
paper_authors._id: 678603c0c054524aa22be7f0
paper_authors.name: Serena Yeung-Levy
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-01-14T06:27:14.763Z
paper_authors._id: 677c8b2e92550a07fcad0f50
paper_authors.avatarUrl: /avatars/2be26e8f25e98cfe5b1d227ee0409cd0.svg
paper_authors.fullname: Serena Yeung-Levy
paper_authors.type: user
paper_authors.user: yeunglevy
paper_title: BIOMEDICA: An Open Biomedical Image-Caption Archive, Dataset, and
    Vision-Language Models Derived from Scientific Literature
paper_summary: The development of vision-language models (VLMs) is driven by large-scale and
  diverse multimodal datasets. However, progress toward generalist biomedical
  VLMs is limited by the lack of annotated, publicly accessible datasets across
  biology and medicine. Existing efforts are restricted to narrow domains,
  missing the full diversity of biomedical knowledge encoded in scientific
  literature. To address this gap, we introduce BIOMEDICA, a scalable,
  open-source framework to extract, annotate, and serialize the entirety of the
  PubMed Central Open Access subset into an easy-to-use, publicly accessible
  dataset.Our framework produces a comprehensive archive with over 24 million
  unique image-text pairs from over 6 million articles. Metadata and
  expert-guided annotations are also provided. We demonstrate the utility and
  accessibility of our resource by releasing BMCA-CLIP, a suite of CLIP-style
  models continuously pre-trained on the BIOMEDICA dataset via streaming,
  eliminating the need to download 27 TB of data locally.On average, our models
  achieve state-of-the-art performance across 40 tasks - spanning pathology,
  radiology, ophthalmology, dermatology, surgery, molecular biology,
  parasitology, and cell biology - excelling in zero-shot classification with a
  6.56% average improvement (as high as 29.8% and 17.5% in dermatology and
  ophthalmology, respectively), and stronger image-text retrieval, all while
  using 10x less compute. To foster reproducibility and collaboration, we release
  our codebase and dataset for the broader research community.
paper_ai_summary: BIOMEDICA is a scalable framework that creates a large, publicly accessible dataset of image-text pairs from PubMed Central, enabling state-of-the-art performance across various biomedical tasks with efficient CLIP-style models.
paper_ai_keywords: vision-language models VLMs multimodal datasets BIOMEDICA PubMed Central Open Access image-text pairs Metadata expert-guided annotations BMCA-CLIP CLIP-style models streaming zero-shot classification image-text retrieval