.
small ones. The only modification is to discard the prediction head (MLP head) and attach a new linear layer. We have explored the direct application of Transformers to image recognition. Unlike prior works using self-attention in computer vision, we do not introduce image-specific inductive biases