MSCPT: Few-shot Whole Slide Image Classification with Multi-scale and Context-focused Prompt Tuning
<|reference_start|>MSCPT: Few-shot Whole Slide Image Classification with Multi-scale and Context-focused Prompt Tuning: Multiple instance learning (MIL) has become a standard paradigm for weakly supervised classification of whole slide images (WSI). However, this paradigm relies on the use of a large number of labelled WSIs for training. The lack of training data and the presence of rare diseases present significant challenges for these methods. Prompt tuning combined with the pre-trained Vision-Language models (VLMs) is an effective solution to the Few-shot Weakly Supervised WSI classification (FSWC) tasks. Nevertheless, applying prompt tuning methods designed for natural images to WSIs presents three significant challenges: 1) These methods fail to fully leverage the prior knowledge from the VLM's text modality; 2) They overlook the essential multi-scale and contextual information in WSIs, leading to suboptimal results; and 3) They lack exploration of instance aggregation methods. To address these problems, we propose a Multi-Scale and Context-focused Prompt Tuning (MSCPT) method for FSWC tasks. Specifically, MSCPT employs the frozen large language model to generate pathological visual language prior knowledge at multi-scale, guiding hierarchical prompt tuning. Additionally, we design a graph prompt tuning module to learn essential contextual information within WSI, and finally, a non-parametric cross-guided instance aggregation module has been introduced to get the WSI-level features. Based on two VLMs, extensive experiments and visualizations on three datasets demonstrated the powerful performance of our MSCPT.<|reference_end|>
@article{han2024mscpt:,
    title={MSCPT: Few-shot Whole Slide Image Classification with Multi-scale and
  Context-focused Prompt Tuning},
    author={Minghao Han, Linhao Qu, Dingkang Yang, Xukun Zhang, Xiaoying Wang,
  Lihua Zhang},
    journal={arXiv preprint arXiv:2408.11505},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.11505},
    primaryClass={cs.CV}
}