OpenMEDLab: An Open-source Platform for Multi-modality Foundation Models in Medicine
<|reference_start|>OpenMEDLab: An Open-source Platform for Multi-modality Foundation Models in Medicine: The emerging trend of advancing generalist artificial intelligence, such as GPTv4 and Gemini, has reshaped the landscape of research (academia and industry) in machine learning and many other research areas. However, domain-specific applications of such foundation models (e.g., in medicine) remain untouched or often at their very early stages. It will require an individual set of transfer learning and model adaptation techniques by further expanding and injecting these models with domain knowledge and data. The development of such technologies could be largely accelerated if the bundle of data, algorithms, and pre-trained foundation models were gathered together and open-sourced in an organized manner. In this work, we present OpenMEDLab, an open-source platform for multi-modality foundation models. It encapsulates not only solutions of pioneering attempts in prompting and fine-tuning large language and vision models for frontline clinical and bioinformatic applications but also building domain-specific foundation models with large-scale multi-modal medical data. Importantly, it opens access to a group of pre-trained foundation models for various medical image modalities, clinical text, protein engineering, etc. Inspiring and competitive results are also demonstrated for each collected approach and model in a variety of benchmarks for downstream tasks. We welcome researchers in the field of medical artificial intelligence to continuously contribute cutting-edge methods and models to OpenMEDLab, which can be accessed via https://github.com/openmedlab.<|reference_end|>
@article{wang2024openmedlab:,
    title={OpenMEDLab: An Open-source Platform for Multi-modality Foundation Models
  in Medicine},
    author={Xiaosong Wang and Xiaofan Zhang and Guotai Wang and Junjun He and
  Zhongyu Li and Wentao Zhu and Yi Guo and Qi Dou and Xiaoxiao Li and Dequan
  Wang and Liang Hong and Qicheng Lao and Tong Ruan and Yukun Zhou and Yixue Li
  and Jie Zhao and Kang Li and Xin Sun and Lifeng Zhu and Shaoting Zhang},
    journal={arXiv preprint arXiv:2402.18028},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.18028},
    primaryClass={cs.CV}
}