UniBiomed: A Universal Foundation Model for Grounded Biomedical Image
  Interpretation
Multi-modal interpretation of biomedical images opens up novel opportunities
in biomedical image analysis. Conventional AI approaches typically rely on
disjointed training, i.e., Large Language Models (LLMs) for clinical text
generation and segmentation models for target extraction, which results in
inflexible real-world deployment and a failure to leverage holistic biomedical
information. To this end, we introduce UniBiomed, the first universal
foundation model for grounded biomedical image interpretation. UniBiomed is
based on a novel integration of Multi-modal Large Language Model (MLLM) and
Segment Anything Model (SAM), which effectively unifies the generation of
clinical texts and the segmentation of corresponding biomedical objects for
grounded interpretation. In this way, UniBiomed is capable of tackling a wide
range of biomedical tasks across ten diverse biomedical imaging modalities. To
develop UniBiomed, we curate a large-scale dataset comprising over 27 million
triplets of images, annotations, and text descriptions across ten imaging
modalities. Extensive validation on 84 internal and external datasets
demonstrated that UniBiomed achieves state-of-the-art performance in
segmentation, disease recognition, region-aware diagnosis, visual question
answering, and report generation. Moreover, unlike previous models that rely on
clinical experts to pre-diagnose images and manually craft precise textual or
visual prompts, UniBiomed can provide automated and end-to-end grounded
interpretation for biomedical image analysis. This represents a novel paradigm
shift in clinical workflows, which will significantly improve diagnostic
efficiency. In summary, UniBiomed represents a novel breakthrough in biomedical
AI, unlocking powerful grounded interpretation capabilities for more accurate
and efficient biomedical image analysis.
68133878475d26db59993ae2
Linshan Wu
claimed_verified
2025-05-02T06:34:46.831Z
65d86ea2685624d5f206d7ec
/avatars/b9bc4c398d5def393bc782e9a7c5e302.svg
Linshan Wu
user
Luffy503
68133878475d26db59993ae3
Yuxiang Nie
admin_assigned
2025-05-05T08:05:30.709Z
65b1d98eb389ca2de13c359b
/avatars/0e8194ab1c7424c6811ec58971337113.svg
NIE Yuxiang
user
JerrryNie
68133878475d26db59993ae4
Sunan He
68133878475d26db59993ae5
Jiaxin Zhuang
68133878475d26db59993ae6
Hao Chen
UniBiomed: A Universal Foundation Model for Grounded Biomedical Image
  Interpretation
Multi-modal interpretation of biomedical images opens up novel opportunities
in biomedical image analysis. Conventional AI approaches typically rely on
disjointed training, i.e., Large Language Models (LLMs) for clinical text
generation and segmentation models for target extraction, which results in
inflexible real-world deployment and a failure to leverage holistic biomedical
information. To this end, we introduce UniBiomed, the first universal
foundation model for grounded biomedical image interpretation. UniBiomed is
based on a novel integration of Multi-modal Large Language Model (MLLM) and
Segment Anything Model (SAM), which effectively unifies the generation of
clinical texts and the segmentation of corresponding biomedical objects for
grounded interpretation. In this way, UniBiomed is capable of tackling a wide
range of biomedical tasks across ten diverse biomedical imaging modalities. To
develop UniBiomed, we curate a large-scale dataset comprising over 27 million
triplets of images, annotations, and text descriptions across ten imaging
modalities. Extensive validation on 84 internal and external datasets
demonstrated that UniBiomed achieves state-of-the-art performance in
segmentation, disease recognition, region-aware diagnosis, visual question
answering, and report generation. Moreover, unlike previous models that rely on
clinical experts to pre-diagnose images and manually craft precise textual or
visual prompts, UniBiomed can provide automated and end-to-end grounded
interpretation for biomedical image analysis. This represents a novel paradigm
shift in clinical workflows, which will significantly improve diagnostic
efficiency. In summary, UniBiomed represents a novel breakthrough in biomedical
AI, unlocking powerful grounded interpretation capabilities for more accurate
and efficient biomedical image analysis.
UniBiomed, a universal foundation model integrating Multi-modal Large Language Model (MLLM) and Segment Anything Model (SAM), provides state-of-the-art performance across various biomedical tasks without requiring pre-diagnoses or manual prompts.
Multi-modal Large Language Model MLLM Segment Anything Model SAM grounded biomedical image interpretation biomedical image analysis clinical text generation segmentation models unified interpretation large-scale dataset segmentation disease recognition region-aware diagnosis visual question answering report generation end-to-end interpretation clinical workflows