LiteGPT: Large Vision-Language Model for Joint Chest X-ray Localization and Classification Task
<|reference_start|>LiteGPT: Large Vision-Language Model for Joint Chest X-ray Localization and Classification Task: Vision-language models have been extensively explored across a wide range of tasks, achieving satisfactory performance; however, their application in medical imaging remains underexplored. In this work, we propose a unified framework - LiteGPT - for the medical imaging. We leverage multiple pre-trained visual encoders to enrich information and enhance the performance of vision-language models. To the best of our knowledge, this is the first study to utilize vision-language models for the novel task of joint localization and classification in medical images. Besides, we are pioneers in providing baselines for disease localization in chest X-rays. Finally, we set new state-of-the-art performance in the image classification task on the well-benchmarked VinDr-CXR dataset. All code and models are publicly available online: https://github.com/leduckhai/LiteGPT<|reference_end|>
@article{le-duc2024litegpt:,
    title={LiteGPT: Large Vision-Language Model for Joint Chest X-ray Localization
  and Classification Task},
    author={Khai Le-Duc, Ryan Zhang, Ngoc Son Nguyen, Tan-Hanh Pham, Anh Dao, Ba
  Hung Ngo, Anh Totti Nguyen, Truong-Son Hy},
    journal={arXiv preprint arXiv:2407.12064},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.12064},
    primaryClass={eess.IV cs.CL cs.CV cs.LG cs.MM}
}