Deep Learning, Artificial Intelligence, Image Classification, Vision Transformer, Transformers.
strategy is used that wields both locality and global dependency. To get downsampled Queries, pooling is used as static local downsampling, 3 × 3 DWCONV is used as learnable local downsampling, and the results are combined and projected into Query dimension. In addition, the attention downsampling