Deep Learning, Artificial Intelligence, Image Classification, Vision Transformer, Small Data.
channel-wise. Then the re-calibrated class token will be concatenated with output patch tokens to restore the token sequence. The process can be formulated as: 1.4. Head-Interacted Multi-Head Self-Attention (HI-MHSA) Head-Interacted Multi-Head Self-Attention (HI-MHSA) In the original MHSA module,