Deep Learning, Artificial Intelligence, Image Classification, Vision Transformer, Transformers.
in DeiT, respectively. The other models vary by different expanding ratios in FFN(r), depths and embedding dimensions. For example, CrossViT-15 has 3 multi-scale encoders, each of which includes 5 regular Transformers, resulting in a total of 15 Transformer encoders. A suffix † means the linear