MedDr: Diagnosis-Guided Bootstrapping for Large-Scale Medical Vision-Language Learning
<|reference_start|>MedDr: Diagnosis-Guided Bootstrapping for Large-Scale Medical Vision-Language Learning: The rapid advancement of large-scale vision-language models has showcased remarkable capabilities across various tasks. However, the lack of extensive and high-quality image-text data in medicine has greatly hindered the development of large-scale medical vision-language models. In this work, we present a diagnosis-guided bootstrapping strategy that exploits both image and label information to construct vision-language datasets. Based on the constructed dataset, we developed MedDr, a generalist foundation model for healthcare capable of handling diverse medical data modalities, including radiology, pathology, dermatology, retinography, and endoscopy. Moreover, during inference, we propose a simple but effective retrieval-augmented medical diagnosis strategy, which enhances the model's generalization ability. Extensive experiments on visual question answering, medical report generation, and medical image diagnosis demonstrate the superiority of our method.<|reference_end|>
@article{he2024gsco:,
    title={GSCo: Towards Generalizable AI in Medicine via Generalist-Specialist
  Collaboration},
    author={Sunan He, Yuxiang Nie, Hongmei Wang, Shu Yang, Yihui Wang, Zhiyuan
  Cai, Zhixuan Chen, Yingxue Xu, Luyang Luo, Huiling Xiang, Xi Lin, Mingxiang
  Wu, Yifan Peng, George Shih, Ziyang Xu, Xian Wu, Qiong Wang, Ronald Cheong
  Kin Chan, Varut Vardhanabhuti, Winnie Chiu Wing Chu, Yefeng Zheng, Pranav
  Rajpurkar, Kang Zhang, Hao Chen},
    journal={arXiv preprint arXiv:2404.15127},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.15127},
    primaryClass={cs.CV cs.CL}
}