Deep Learning, Computer Vision, Cnn, Transformers, AI.
the unique representation properties observed in ViTs. Comparison of ViT and ResNet Layers: The authors create a heatmap comparing all ViT layers to all ResNet layers using Center Kernel Alignment (CKA) similarity. The results indicate that the lower half of ResNet layers resembles the bottom