Deep Learning, Artificial Intelligence, Image Classification, Vision Transformer, Transformers.
for various design choices. By improving each component one by one, the performance becomes better. The components are described below. (a) EfficientFormerV1 as baseline. (b) Unified FFN. (c) MHSA improvements. (d)&(e) Attention on higher resolution. (f) Attention downsampling. 1.1. Token Mixers