title: HealthGPT: A Medical Large Vision-Language Model for Unifying
    Comprehension and Generation via Heterogeneous Knowledge Adaptation
summary: We present HealthGPT, a powerful Medical Large Vision-Language Model
  (Med-LVLM) that integrates medical visual comprehension and generation
  capabilities within a unified autoregressive paradigm. Our bootstrapping
  philosophy is to progressively adapt heterogeneous comprehension and generation
  knowledge to pre-trained large language models (LLMs). This is achieved through
  a novel heterogeneous low-rank adaptation (H-LoRA) technique, which is
  complemented by a tailored hierarchical visual perception approach and a
  three-stage learning strategy. To effectively learn the HealthGPT, we devise a
  comprehensive medical domain-specific comprehension and generation dataset
  called VL-Health. Experimental results demonstrate exceptional performance and
  scalability of HealthGPT in medical visual unified tasks. Our project can be
  accessed at https://github.com/DCDmllm/HealthGPT.
paper_authors._id: 67b55078a64445f58c771d84
paper_authors.name: Tianwei Lin
paper_authors._id: 67b55078a64445f58c771d85
paper_authors.name: Wenqiao Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-19T14:40:41.324Z
paper_authors._id: 65fc18edfb66882aba4d548e
paper_authors.avatarUrl: /avatars/f70d47fe4aba98b5a5cd64f7e002dfd2.svg
paper_authors.fullname: wenqiao
paper_authors.type: user
paper_authors.user: wannature
paper_authors._id: 67b55078a64445f58c771d86
paper_authors.name: Sijing Li
paper_authors._id: 67b55078a64445f58c771d87
paper_authors.name: Yuqian Yuan
paper_authors._id: 67b55078a64445f58c771d88
paper_authors.name: Binhe Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-19T14:41:15.907Z
paper_authors._id: 648ad056dc6a5b4b88306ae2
paper_authors.avatarUrl: /avatars/66a47a11626c05b706de33a3184182e9.svg
paper_authors.fullname: yu
paper_authors.type: user
paper_authors.user: binheyu1991
paper_authors._id: 67b55078a64445f58c771d89
paper_authors.name: Haoyuan Li
paper_authors._id: 67b55078a64445f58c771d8a
paper_authors.name: Wanggui He
paper_authors._id: 67b55078a64445f58c771d8b
paper_authors.name: Hao Jiang
paper_authors._id: 67b55078a64445f58c771d8c
paper_authors.name: Mengze Li
paper_authors._id: 67b55078a64445f58c771d8d
paper_authors.name: Xiaohui Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-19T14:42:09.080Z
paper_authors._id: 635156f1f74cdcca6f7acf70
paper_authors.avatarUrl: /avatars/133e88b30255425f6da7777737f91e81.svg
paper_authors.fullname: Xiaohui Song
paper_authors.type: user
paper_authors.user: fpcsong
paper_authors._id: 67b55078a64445f58c771d8e
paper_authors.name: Siliang Tang
paper_authors._id: 67b55078a64445f58c771d8f
paper_authors.name: Jun Xiao
paper_authors._id: 67b55078a64445f58c771d90
paper_authors.name: Hui Lin
paper_authors._id: 67b55078a64445f58c771d91
paper_authors.name: Yueting Zhuang
paper_authors._id: 67b55078a64445f58c771d92
paper_authors.name: Beng Chin Ooi
paper_title: HealthGPT: A Medical Large Vision-Language Model for Unifying
    Comprehension and Generation via Heterogeneous Knowledge Adaptation
paper_summary: We present HealthGPT, a powerful Medical Large Vision-Language Model
  (Med-LVLM) that integrates medical visual comprehension and generation
  capabilities within a unified autoregressive paradigm. Our bootstrapping
  philosophy is to progressively adapt heterogeneous comprehension and generation
  knowledge to pre-trained large language models (LLMs). This is achieved through
  a novel heterogeneous low-rank adaptation (H-LoRA) technique, which is
  complemented by a tailored hierarchical visual perception approach and a
  three-stage learning strategy. To effectively learn the HealthGPT, we devise a
  comprehensive medical domain-specific comprehension and generation dataset
  called VL-Health. Experimental results demonstrate exceptional performance and
  scalability of HealthGPT in medical visual unified tasks. Our project can be
  accessed at https://github.com/DCDmllm/HealthGPT.
paper_ai_keywords: Medical Large Vision-Language Model (Med-LVLM) autoregressive paradigm heterogeneous low-Rank Adaptation (H-LoRA) hierarchical visual perception three-stage learning strategy VL-Health medical visual Unified tasks