Deep Learning, Artificial Intelligence, Image Classification, Vision Transformer, Resnet.
representations between lower and higher layers (iii) the highest layers of ViT have quite different representations to ResNet. 2.2. Local and Global Information in Layer Representations Atttention Head Mean Distances Even in the lowest layers of ViT, self-attention layers have a mix of local heads