Deep Learning, Artificial Intelligence, Image Classification, Vision Transformer, Small Data.
into h parts. Each part contains d channels, where D=d×h. All the separated tokens are averaged in their own parts. Thus we get totally h tokens and each one is d-dimensional. All such intermediate tokens will be projected into D-dimension again, resulting in h head tokens in total. Finally, they