A foundation model utilizing chest CT volumes and radiology reports for supervised-level zero-shot detection of abnormalities
<|reference_start|>A foundation model utilizing chest CT volumes and radiology reports for supervised-level zero-shot detection of abnormalities: A major challenge in computational research in 3D medical imaging is the lack of comprehensive datasets. Addressing this issue, our study introduces CT-RATE, the first 3D medical imaging dataset that pairs images with textual reports. CT-RATE consists of 25,692 non-contrast chest CT volumes, expanded to 50,188 through various reconstructions, from 21,304 unique patients, along with corresponding radiology text reports. Leveraging CT-RATE, we developed CT-CLIP, a CT-focused contrastive language-image pre-training framework. As a versatile, self-supervised model, CT-CLIP is designed for broad application and does not require task-specific training. Remarkably, CT-CLIP outperforms state-of-the-art, fully supervised methods in multi-abnormality detection across all key metrics, thus eliminating the need for manual annotation. We also demonstrate its utility in case retrieval, whether using imagery or textual queries, thereby advancing knowledge dissemination. The open-source release of CT-RATE and CT-CLIP marks a significant advancement in medical AI, enhancing 3D imaging analysis and fostering innovation in healthcare.<|reference_end|>
@article{hamamci2024developing,
    title={Developing Generalist Foundation Models from a Multimodal Dataset for 3D
  Computed Tomography},
    author={Ibrahim Ethem Hamamci, Sezgin Er, Furkan Almas, Ayse Gulnihan Simsek,
  Sevval Nil Esirgun, Irem Dogan, Muhammed Furkan Dasdelen, Omer Faruk Durugol,
  Bastian Wittmann, Tamaz Amiranashvili, Enis Simsar, Mehmet Simsar, Emine
  Bensu Erdemir, Abdullah Alanbay, Anjany Sekuboyina, Berkan Lafci, Christian
  Bluethgen, Mehmet Kemal Ozdemir, Bjoern Menze},
    journal={arXiv preprint arXiv:2403.17834},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.17834},
    primaryClass={cs.CV}
}