  Pathological captioning of Whole Slide Images (WSIs), though is essential in
computer-aided pathological diagnosis, has rarely been studied due to the
limitations in datasets and model training efficacy. In this paper, we propose
a new paradigm Subtype-guided Masked Transformer (SGMT) for pathological
captioning based on Transformers, which treats a WSI as a sequence of sparse
patches and generates an overall caption sentence from the sequence. An
accompanying subtype prediction is introduced into SGMT to guide the training
process and enhance the captioning accuracy. We also present an Asymmetric
Masked Mechansim approach to tackle the large size constraint of pathological
image captioning, where the numbers of sequencing patches in SGMT are sampled
differently in the training and inferring phases, respectively. Experiments on
the PatchGastricADC22 dataset demonstrate that our approach effectively adapts
to the task with a transformer-based model and achieves superior performance
than traditional RNN-based methods. Our codes are to be made available for
further research and development.

Computer Vision and Pattern Recognition;Artificial Intelligence