Deep Learning, Artificial Intelligence, Image Classification, Vision Transformer, Small Data.
each attention head has not interacted with others. Under this circumstance, the representation in each channel group is too weak for recognition especially when it is lack of training data. (a) Head Token Generation: In HI-MHSA, each D-dimensional token, including class token, will be reshaped