  Modern deep neural networks have achieved great successes in medical image
analysis. However, the features captured by convolutional neural networks
(CNNs) or Transformers tend to be optimized for pixel intensities and neglect
key anatomical structures such as connected components and loops. In this
paper, we propose a persistent homology guided approach (PHG-Net) that explores
topological features of objects for medical image classification. For an input
image, we first compute its cubical persistence diagram and extract topological
features into a vector representation using a small neural network (called the
PH module). The extracted topological features are then incorporated into the
feature map generated by CNN or Transformer for feature fusion. The PH module
is lightweight and capable of integrating topological features into any CNN or
Transformer architectures in an end-to-end fashion. We evaluate our PHG-Net on
three public datasets and demonstrate its considerable improvements on the
target classification tasks over state-of-the-art methods.

Computer Vision and Pattern Recognition;Image and Video Processing