Deep Learning, Artificial Intelligence, Image Classification, Vision Transformer, Small Data.
embedding is called Sequential Overlapping Patch Embedding (SOPE), which contains several successive convolution layers of 3×3 convolution with stride s=2, Batch Normalization and GELU activation. The relation between the number of convolution layers and the patch size is P=2^k. SOPE is able to