MamMIL: Multiple Instance Learning for Whole Slide Images with State Space Models
<|reference_start|>MamMIL: Multiple Instance Learning for Whole Slide Images with State Space Models: Recently, pathological diagnosis, the gold standard for cancer diagnosis, has achieved superior performance by combining the Transformer with the multiple instance learning (MIL) framework using whole slide images (WSIs). However, the giga-pixel nature of WSIs poses a great challenge for the quadratic-complexity self-attention mechanism in Transformer to be applied in MIL. Existing studies usually use linear attention to improve computing efficiency but inevitably bring performance bottlenecks. To tackle this challenge, we propose a MamMIL framework for WSI classification by cooperating the selective structured state space model (i.e., Mamba) with MIL for the first time, enabling the modeling of instance dependencies while maintaining linear complexity. Specifically, to solve the problem that Mamba can only conduct unidirectional one-dimensional (1D) sequence modeling, we innovatively introduce a bidirectional state space model and a 2D context-aware block to enable MamMIL to learn the bidirectional instance dependencies with 2D spatial relationships. Experiments on two datasets show that MamMIL can achieve advanced classification performance with smaller memory footprints than the state-of-the-art MIL frameworks based on the Transformer. The code will be open-sourced if accepted.<|reference_end|>
@article{fang2024mammil:,
    title={MamMIL: Multiple Instance Learning for Whole Slide Images with State
  Space Models},
    author={Zijie Fang, Yifeng Wang, Ye Zhang, Zhi Wang, Jian Zhang, Xiangyang Ji,
  Yongbing Zhang},
    journal={arXiv preprint arXiv:2403.05160},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.05160},
    primaryClass={cs.CV}
}