BiMediX2: Bio-Medical EXpert LMM for Diverse Medical Modalities
This paper introduces BiMediX2, a bilingual (Arabic-English) Bio-Medical
EXpert Large Multimodal Model (LMM) with a unified architecture that integrates
text and visual modalities, enabling advanced image understanding and medical
applications. BiMediX2 leverages the Llama3.1 architecture and integrates text
and visual capabilities to facilitate seamless interactions in both English and
Arabic, supporting text-based inputs and multi-turn conversations involving
medical images. The model is trained on an extensive bilingual healthcare
dataset consisting of 1.6M samples of diverse medical interactions for both
text and image modalities, mixed in Arabic and English. We also propose the
first bilingual GPT-4o based medical LMM benchmark named BiMed-MBench. BiMediX2
is benchmarked on both text-based and image-based tasks, achieving
state-of-the-art performance across several medical benchmarks. It outperforms
recent state-of-the-art models in medical LLM evaluation benchmarks. Our model
also sets a new benchmark in multimodal medical evaluations with over 9%
improvement in English and over 20% in Arabic evaluations. Additionally, it
surpasses GPT-4 by around 9% in UPHILL factual accuracy evaluations and excels
in various medical Visual Question Answering, Report Generation, and Report
Summarization tasks. The project page including source code and the trained
model, is available at https://github.com/mbzuai-oryx/BiMediX2.
675ee6efe6d71589c3c62d6e
Sahal Shaji Mullappilly
admin_assigned
2024-12-16T10:03:51.461Z
62e23c7f555a866437a53cd0
https://cdn-avatars.huggingface.co/v1/production/uploads/62e23c7f555a866437a53cd0/UaAsYZQXuwb4NSG5WnvdG.jpeg
Sahal Shaji
user
sahalshajim
675ee6efe6d71589c3c62d6f
Mohammed Irfan Kurpath
claimed_verified
2024-12-16T13:25:34.125Z
650289dbc130d99814b34dc5
/avatars/ff0cf5add144cd79c41a255f41f34efb.svg
K Mohammed Irfan
user
k-m-irfan
675ee6efe6d71589c3c62d70
Sara Pieri
admin_assigned
2024-12-16T10:02:01.391Z
63fe2f588b3c5087ff8721bf
https://cdn-avatars.huggingface.co/v1/production/uploads/1677602580834-noauth.jpeg
Sara Pieri
user
HuggingSara
675ee6efe6d71589c3c62d71
Saeed Yahya Alseiari
675ee6efe6d71589c3c62d72
Shanavas Cholakkal
675ee6efe6d71589c3c62d73
Khaled Aldahmani
675ee6efe6d71589c3c62d74
Fahad Khan
675ee6efe6d71589c3c62d75
Rao Anwer
675ee6efe6d71589c3c62d76
Salman Khan
675ee6efe6d71589c3c62d77
Timothy Baldwin
675ee6efe6d71589c3c62d78
Hisham Cholakkal
admin_assigned
2024-12-16T10:00:42.925Z
654a5f4f9b8bd6406d45bb46
/avatars/ac0d7eef62cd98a280b162cf7896b1a2.svg
Hisham Cholakkal
user
hishamcholakkal
BiMediX2: Bio-Medical EXpert LMM for Diverse Medical Modalities
This paper introduces BiMediX2, a bilingual (Arabic-English) Bio-Medical
EXpert Large Multimodal Model (LMM) with a unified architecture that integrates
text and visual modalities, enabling advanced image understanding and medical
applications. BiMediX2 leverages the Llama3.1 architecture and integrates text
and visual capabilities to facilitate seamless interactions in both English and
Arabic, supporting text-based inputs and multi-turn conversations involving
medical images. The model is trained on an extensive bilingual healthcare
dataset consisting of 1.6M samples of diverse medical interactions for both
text and image modalities, mixed in Arabic and English. We also propose the
first bilingual GPT-4o based medical LMM benchmark named BiMed-MBench. BiMediX2
is benchmarked on both text-based and image-based tasks, achieving
state-of-the-art performance across several medical benchmarks. It outperforms
recent state-of-the-art models in medical LLM evaluation benchmarks. Our model
also sets a new benchmark in multimodal medical evaluations with over 9%
improvement in English and over 20% in Arabic evaluations. Additionally, it
surpasses GPT-4 by around 9% in UPHILL factual accuracy evaluations and excels
in various medical Visual Question Answering, Report Generation, and Report
Summarization tasks. The project page including source code and the trained
model, is available at https://github.com/mbzuai-oryx/BiMediX2.
BiMediX2, a bilingual Arabic-English biomedical multimodal model, achieves state-of-the-art performance in medical applications across text-based and image-based tasks.
BiMediX2 Bio-Medical EXpert Large Multimodal Model Llama3.1 biomedical multimodal model BiMed-MBench medical LMM benchmark UPHILL factual accuracy medical Visual Question Answering Report Generation Report Summarization