  The foundation models based on pre-training technology have significantly
advanced artificial intelligence from theoretical to practical applications.
These models have facilitated the feasibility of computer-aided diagnosis for
widespread use. Medical contrastive vision-language pre-training, which does
not require human annotations, is an effective approach for guiding
representation learning using description information in diagnostic reports.
However, the effectiveness of pre-training is limited by the large-scale
semantic overlap and shifting problems in medical field. To address these
issues, we propose the Knowledge-Boosting Contrastive Vision-Language
Pre-training framework (KoBo), which integrates clinical knowledge into the
learning of vision-language semantic consistency. The framework uses an
unbiased, open-set sample-wise knowledge representation to measure negative
sample noise and supplement the correspondence between vision-language mutual
information and clinical knowledge. Extensive experiments validate the effect
of our framework on eight tasks including classification, segmentation,
retrieval, and semantic relatedness, achieving comparable or better performance
with the zero-shot or few-shot settings. Our code is open on
https://github.com/ChenXiaoFei-CS/KoBo.

Computer Vision and Pattern Recognition;Machine Learning