Benchmarking Image Transformers for Prostate Cancer Detection from Ultrasound Data
<|reference_start|>Benchmarking Image Transformers for Prostate Cancer Detection from Ultrasound Data: PURPOSE: Deep learning methods for classifying prostate cancer (PCa) in ultrasound images typically employ convolutional networks (CNNs) to detect cancer in small regions of interest (ROI) along a needle trace region. However, this approach suffers from weak labelling, since the ground-truth histopathology labels do not describe the properties of individual ROIs. Recently, multi-scale approaches have sought to mitigate this issue by combining the context awareness of transformers with a CNN feature extractor to detect cancer from multiple ROIs using multiple-instance learning (MIL). In this work, we present a detailed study of several image transformer architectures for both ROI-scale and multi-scale classification, and a comparison of the performance of CNNs and transformers for ultrasound-based prostate cancer classification. We also design a novel multi-objective learning strategy that combines both ROI and core predictions to further mitigate label noise. METHODS: We evaluate 3 image transformers on ROI-scale cancer classification, then use the strongest model to tune a multi-scale classifier with MIL. We train our MIL models using our novel multi-objective learning strategy and compare our results to existing baselines. RESULTS: We find that for both ROI-scale and multi-scale PCa detection, image transformer backbones lag behind their CNN counterparts. This deficit in performance is even more noticeable for larger models. When using multi-objective learning, we can improve performance of MIL, with a 77.9% AUROC, a sensitivity of 75.9%, and a specificity of 66.3%. CONCLUSION: Convolutional networks are better suited for modelling sparse datasets of prostate ultrasounds, producing more robust features than transformers in PCa detection. Multi-scale methods remain the best architecture for this task, with multi-objective learning presenting an effective way to improve performance.<|reference_end|>
@article{harmanani2024benchmarking,
    title={Benchmarking Image Transformers for Prostate Cancer Detection from
  Ultrasound Data},
    author={Mohamed Harmanani, Paul F. R. Wilson, Fahimeh Fooladgar, Amoon Jamzad,
  Mahdi Gilany, Minh Nguyen Nhat To, Brian Wodlinger, Purang Abolmaesumi,
  Parvin Mousavi},
    journal={Proc. SPIE 12928, Medical Imaging 2024: Image-Guided Procedures,
  Robotic Interventions, and Modeling, 1292815 (29 March 2024)},
    year={2024},
    doi={10.1117/12.3006049},
    archivePrefix={arXiv},
    eprint={2403.18233},
    primaryClass={eess.IV cs.CV cs.LG q-bio.TO}
}