title: BiMediX2: Bio-Medical EXpert LMM for Diverse Medical Modalities
summary: This paper introduces BiMediX2, a bilingual (Arabic-English) Bio-Medical
  EXpert Large Multimodal Model (LMM) with a unified architecture that integrates
  text and visual modalities, enabling advanced image understanding and medical
  applications. BiMediX2 leverages the Llama3.1 architecture and integrates text
  and visual capabilities to facilitate seamless interactions in both English and
  Arabic, supporting text-based inputs and multi-turn conversations involving
  medical images. The model is trained on an extensive bilingual healthcare
  dataset consisting of 1.6M samples of diverse medical interactions for both
  text and image modalities, mixed in Arabic and English. We also propose the
  first bilingual GPT-4o based medical LMM benchmark named BiMed-MBench. BiMediX2
  is benchmarked on both text-based and image-based tasks, achieving
  state-of-the-art performance across several medical benchmarks. It outperforms
  recent state-of-the-art models in medical LLM evaluation benchmarks. Our model
  also sets a new benchmark in multimodal medical evaluations with over 9%
  improvement in English and over 20% in Arabic evaluations. Additionally, it
  surpasses GPT-4 by around 9% in UPHILL factual accuracy evaluations and excels
  in various medical Visual Question Answering, Report Generation, and Report
  Summarization tasks. The project page including source code and the trained
  model, is available at https://github.com/mbzuai-oryx/BiMediX2.
paper_authors._id: 675ee6efe6d71589c3c62d6e
paper_authors.name: Sahal Shaji Mullappilly
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T10:03:51.461Z
paper_authors._id: 62e23c7f555a866437a53cd0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62e23c7f555a866437a53cd0/UaAsYZQXuwb4NSG5WnvdG.jpeg
paper_authors.fullname: Sahal Shaji
paper_authors.type: user
paper_authors.user: sahalshajim
paper_authors._id: 675ee6efe6d71589c3c62d6f
paper_authors.name: Mohammed Irfan Kurpath
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-16T13:25:34.125Z
paper_authors._id: 650289dbc130d99814b34dc5
paper_authors.avatarUrl: /avatars/ff0cf5add144cd79c41a255f41f34efb.svg
paper_authors.fullname: K Mohammed Irfan
paper_authors.type: user
paper_authors.user: k-m-irfan
paper_authors._id: 675ee6efe6d71589c3c62d70
paper_authors.name: Sara Pieri
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T10:02:01.391Z
paper_authors._id: 63fe2f588b3c5087ff8721bf
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1677602580834-noauth.jpeg
paper_authors.fullname: Sara Pieri
paper_authors.type: user
paper_authors.user: HuggingSara
paper_authors._id: 675ee6efe6d71589c3c62d71
paper_authors.name: Saeed Yahya Alseiari
paper_authors._id: 675ee6efe6d71589c3c62d72
paper_authors.name: Shanavas Cholakkal
paper_authors._id: 675ee6efe6d71589c3c62d73
paper_authors.name: Khaled Aldahmani
paper_authors._id: 675ee6efe6d71589c3c62d74
paper_authors.name: Fahad Khan
paper_authors._id: 675ee6efe6d71589c3c62d75
paper_authors.name: Rao Anwer
paper_authors._id: 675ee6efe6d71589c3c62d76
paper_authors.name: Salman Khan
paper_authors._id: 675ee6efe6d71589c3c62d77
paper_authors.name: Timothy Baldwin
paper_authors._id: 675ee6efe6d71589c3c62d78
paper_authors.name: Hisham Cholakkal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T10:00:42.925Z
paper_authors._id: 654a5f4f9b8bd6406d45bb46
paper_authors.avatarUrl: /avatars/ac0d7eef62cd98a280b162cf7896b1a2.svg
paper_authors.fullname: Hisham Cholakkal
paper_authors.type: user
paper_authors.user: hishamcholakkal
paper_title: BiMediX2: Bio-Medical EXpert LMM for Diverse Medical Modalities
paper_summary: This paper introduces BiMediX2, a bilingual (Arabic-English) Bio-Medical
  EXpert Large Multimodal Model (LMM) with a unified architecture that integrates
  text and visual modalities, enabling advanced image understanding and medical
  applications. BiMediX2 leverages the Llama3.1 architecture and integrates text
  and visual capabilities to facilitate seamless interactions in both English and
  Arabic, supporting text-based inputs and multi-turn conversations involving
  medical images. The model is trained on an extensive bilingual healthcare
  dataset consisting of 1.6M samples of diverse medical interactions for both
  text and image modalities, mixed in Arabic and English. We also propose the
  first bilingual GPT-4o based medical LMM benchmark named BiMed-MBench. BiMediX2
  is benchmarked on both text-based and image-based tasks, achieving
  state-of-the-art performance across several medical benchmarks. It outperforms
  recent state-of-the-art models in medical LLM evaluation benchmarks. Our model
  also sets a new benchmark in multimodal medical evaluations with over 9%
  improvement in English and over 20% in Arabic evaluations. Additionally, it
  surpasses GPT-4 by around 9% in UPHILL factual accuracy evaluations and excels
  in various medical Visual Question Answering, Report Generation, and Report
  Summarization tasks. The project page including source code and the trained
  model, is available at https://github.com/mbzuai-oryx/BiMediX2.
paper_ai_summary: BiMediX2, a bilingual Arabic-English biomedical multimodal model, achieves state-of-the-art performance in medical applications across text-based and image-based tasks.
paper_ai_keywords: BiMediX2 Bio-Medical EXpert Large Multimodal Model Llama3.1 biomedical multimodal model BiMed-MBench medical LMM benchmark UPHILL factual accuracy medical Visual Question Answering Report Generation Report Summarization