Deep Learning, Artificial Intelligence, Image Classification, Vision Transformer, Transformers.
patch embeddingis substituted by three convolutional layers as the patch tokenizer, similar to ViT. 4. Results 4.1. Comparison with DeiT Comparisons with DeiT baseline on ImageNet1K. CrossViT improves DeiT-Ti, DeiT-S and DeiT-B by 1.2%, 1.2% and 0.4% points respectively. 4.2. SOTA Comparisons