Deep Learning, Artificial Intelligence, Image Classification, Vision Transformer, Transformers.
vs. Feed Forward Network Unified FFN PoolFormer and EfficientFormer employ 3 × 3 average pooling layers (Fig. 2(a)) as local token mixer. Replacing these layers with depth-wise convolutions (DWCONV) of the same kernel size does not introduce latency overhead, while the performance is improved by