Deep Learning, Artificial Intelligence, Image Classification, Vision Transformer, Small Data.
ViT is formed by two fully-connected layers and GELU. DAFF integrates depth-wise convolution (DWCONV) from MobileNetV1 in FFN. With the inductive bias brought by depth-wise convolution, the model is forced to capture neighboring features, solving the problem on spatial view. It greatly reduces the