title: UniBiomed: A Universal Foundation Model for Grounded Biomedical Image
    Interpretation
summary: Multi-modal interpretation of biomedical images opens up novel opportunities
  in biomedical image analysis. Conventional AI approaches typically rely on
  disjointed training, i.e., Large Language Models (LLMs) for clinical text
  generation and segmentation models for target extraction, which results in
  inflexible real-world deployment and a failure to leverage holistic biomedical
  information. To this end, we introduce UniBiomed, the first universal
  foundation model for grounded biomedical image interpretation. UniBiomed is
  based on a novel integration of Multi-modal Large Language Model (MLLM) and
  Segment Anything Model (SAM), which effectively unifies the generation of
  clinical texts and the segmentation of corresponding biomedical objects for
  grounded interpretation. In this way, UniBiomed is capable of tackling a wide
  range of biomedical tasks across ten diverse biomedical imaging modalities. To
  develop UniBiomed, we curate a large-scale dataset comprising over 27 million
  triplets of images, annotations, and text descriptions across ten imaging
  modalities. Extensive validation on 84 internal and external datasets
  demonstrated that UniBiomed achieves state-of-the-art performance in
  segmentation, disease recognition, region-aware diagnosis, visual question
  answering, and report generation. Moreover, unlike previous models that rely on
  clinical experts to pre-diagnose images and manually craft precise textual or
  visual prompts, UniBiomed can provide automated and end-to-end grounded
  interpretation for biomedical image analysis. This represents a novel paradigm
  shift in clinical workflows, which will significantly improve diagnostic
  efficiency. In summary, UniBiomed represents a novel breakthrough in biomedical
  AI, unlocking powerful grounded interpretation capabilities for more accurate
  and efficient biomedical image analysis.
paper_authors._id: 68133878475d26db59993ae2
paper_authors.name: Linshan Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-02T06:34:46.831Z
paper_authors._id: 65d86ea2685624d5f206d7ec
paper_authors.avatarUrl: /avatars/b9bc4c398d5def393bc782e9a7c5e302.svg
paper_authors.fullname: Linshan Wu
paper_authors.type: user
paper_authors.user: Luffy503
paper_authors._id: 68133878475d26db59993ae3
paper_authors.name: Yuxiang Nie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T08:05:30.709Z
paper_authors._id: 65b1d98eb389ca2de13c359b
paper_authors.avatarUrl: /avatars/0e8194ab1c7424c6811ec58971337113.svg
paper_authors.fullname: NIE Yuxiang
paper_authors.type: user
paper_authors.user: JerrryNie
paper_authors._id: 68133878475d26db59993ae4
paper_authors.name: Sunan He
paper_authors._id: 68133878475d26db59993ae5
paper_authors.name: Jiaxin Zhuang
paper_authors._id: 68133878475d26db59993ae6
paper_authors.name: Hao Chen
paper_title: UniBiomed: A Universal Foundation Model for Grounded Biomedical Image
    Interpretation
paper_summary: Multi-modal interpretation of biomedical images opens up novel opportunities
  in biomedical image analysis. Conventional AI approaches typically rely on
  disjointed training, i.e., Large Language Models (LLMs) for clinical text
  generation and segmentation models for target extraction, which results in
  inflexible real-world deployment and a failure to leverage holistic biomedical
  information. To this end, we introduce UniBiomed, the first universal
  foundation model for grounded biomedical image interpretation. UniBiomed is
  based on a novel integration of Multi-modal Large Language Model (MLLM) and
  Segment Anything Model (SAM), which effectively unifies the generation of
  clinical texts and the segmentation of corresponding biomedical objects for
  grounded interpretation. In this way, UniBiomed is capable of tackling a wide
  range of biomedical tasks across ten diverse biomedical imaging modalities. To
  develop UniBiomed, we curate a large-scale dataset comprising over 27 million
  triplets of images, annotations, and text descriptions across ten imaging
  modalities. Extensive validation on 84 internal and external datasets
  demonstrated that UniBiomed achieves state-of-the-art performance in
  segmentation, disease recognition, region-aware diagnosis, visual question
  answering, and report generation. Moreover, unlike previous models that rely on
  clinical experts to pre-diagnose images and manually craft precise textual or
  visual prompts, UniBiomed can provide automated and end-to-end grounded
  interpretation for biomedical image analysis. This represents a novel paradigm
  shift in clinical workflows, which will significantly improve diagnostic
  efficiency. In summary, UniBiomed represents a novel breakthrough in biomedical
  AI, unlocking powerful grounded interpretation capabilities for more accurate
  and efficient biomedical image analysis.
paper_ai_summary: UniBiomed, a universal foundation model integrating Multi-modal Large Language Model (MLLM) and Segment Anything Model (SAM), provides state-of-the-art performance across various biomedical tasks without requiring pre-diagnoses or manual prompts.
paper_ai_keywords: Multi-modal Large Language Model MLLM Segment Anything Model SAM grounded biomedical image interpretation biomedical image analysis clinical text generation segmentation models unified interpretation large-scale dataset segmentation disease recognition region-aware diagnosis visual question answering report generation end-to-end interpretation clinical workflows