title: MedAgent-Pro: Towards Multi-modal Evidence-based Medical Diagnosis via
    Reasoning Agentic Workflow
summary: Developing reliable AI systems to assist human clinicians in multi-modal
  medical diagnosis has long been a key objective for researchers. Recently,
  Multi-modal Large Language Models (MLLMs) have gained significant attention and
  achieved success across various domains. With strong reasoning capabilities and
  the ability to perform diverse tasks based on user instructions, they hold
  great potential for enhancing medical diagnosis. However, directly applying
  MLLMs to the medical domain still presents challenges. They lack detailed
  perception of visual inputs, limiting their ability to perform quantitative
  image analysis, which is crucial for medical diagnostics. Additionally, MLLMs
  often exhibit hallucinations and inconsistencies in reasoning, whereas clinical
  diagnoses must adhere strictly to established criteria. To address these
  challenges, we propose MedAgent-Pro, an evidence-based reasoning agentic system
  designed to achieve reliable, explainable, and precise medical diagnoses. This
  is accomplished through a hierarchical workflow: at the task level,
  knowledge-based reasoning generate reliable diagnostic plans for specific
  diseases following retrieved clinical criteria. While at the case level,
  multiple tool agents process multi-modal inputs, analyze different indicators
  according to the plan, and provide a final diagnosis based on both quantitative
  and qualitative evidence. Comprehensive experiments on both 2D and 3D medical
  diagnosis tasks demonstrate the superiority and effectiveness of MedAgent-Pro,
  while case studies further highlight its reliability and interpretability. The
  code is available at https://github.com/jinlab-imvr/MedAgent-Pro.
paper_authors._id: 67ea0ede7b856e8fa8ff50d0
paper_authors.name: Ziyue Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-31T09:59:13.929Z
paper_authors._id: 65be4d7d5e342a230dc19a54
paper_authors.avatarUrl: /avatars/04ba16980da94954d811032f0091212f.svg
paper_authors.fullname: Ziyue Wang
paper_authors.type: user
paper_authors.user: ZiyueWang
paper_authors._id: 67ea0ede7b856e8fa8ff50d1
paper_authors.name: Junde Wu
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-03-31T03:41:20.293Z
paper_authors._id: 6317257fc92fd6fee317ff7c
paper_authors.avatarUrl: /avatars/2f460a2f28562c987becb2acad8d93e7.svg
paper_authors.fullname: Junde Wu
paper_authors.type: user
paper_authors.user: morson
paper_authors._id: 67ea0ede7b856e8fa8ff50d2
paper_authors.name: Chang Han Low
paper_authors._id: 67ea0ede7b856e8fa8ff50d3
paper_authors.name: Yueming Jin
paper_title: MedAgent-Pro: Towards Multi-modal Evidence-based Medical Diagnosis via
    Reasoning Agentic Workflow
paper_summary: Developing reliable AI systems to assist human clinicians in multi-modal
  medical diagnosis has long been a key objective for researchers. Recently,
  Multi-modal Large Language Models (MLLMs) have gained significant attention and
  achieved success across various domains. With strong reasoning capabilities and
  the ability to perform diverse tasks based on user instructions, they hold
  great potential for enhancing medical diagnosis. However, directly applying
  MLLMs to the medical domain still presents challenges. They lack detailed
  perception of visual inputs, limiting their ability to perform quantitative
  image analysis, which is crucial for medical diagnostics. Additionally, MLLMs
  often exhibit hallucinations and inconsistencies in reasoning, whereas clinical
  diagnoses must adhere strictly to established criteria. To address these
  challenges, we propose MedAgent-Pro, an evidence-based reasoning agentic system
  designed to achieve reliable, explainable, and precise medical diagnoses. This
  is accomplished through a hierarchical workflow: at the task level,
  knowledge-based reasoning generate reliable diagnostic plans for specific
  diseases following retrieved clinical criteria. While at the case level,
  multiple tool agents process multi-modal inputs, analyze different indicators
  according to the plan, and provide a final diagnosis based on both quantitative
  and qualitative evidence. Comprehensive experiments on both 2D and 3D medical
  diagnosis tasks demonstrate the superiority and effectiveness of MedAgent-Pro,
  while case studies further highlight its reliability and interpretability. The
  code is available at https://github.com/jinlab-imvr/MedAgent-Pro.
paper_ai_summary: MedAgent-Pro addresses challenges in multi-modal medical diagnosis by combining knowledge-based reasoning with tool agents for precise and explainable diagnoses.
paper_ai_keywords: Multi-modal Large Language Models MLMLs evidence-based reasoning enigmatic reasoning clinical criteria hierarchical workflow knowledge-based reasoning tool agents multi-modal inputs quantitative analysis qualitative evidence