Deep Learning, Artificial Intelligence, Image Classification, Vision Transformer, Small Data.
performance gap when training from scratch on small datasets, and converges faster than standard CNNs. Also, similar mechanism as SE module from SENet is used. Xc, Xp denote class token and patch tokens respectively. Specifically, class token is split from sequence as Xc, before the projection