Deep Learning, Artificial Intelligence, Image Classification, Vision Transformer, Small Data.
layers. The remaining patch tokens Xp goes through a depth-wise integrated multi-layer perceptron with a shortcut inside. The output patch tokens will then be averaged into a weight vector W. After the squeeze-excitation operation, the output weight vector will be multiplied with class token