Deep Learning, Artificial Intelligence, Image Classification, Vision Transformer, Transformers.
Second, communications between attention heads are enabled by adding fully connected layers across head dimensions [63] that are shown as Talking Head in Fig. 2(c). With these modifications, the performance is further boosted to 80.8% with similar parameters and latency compared to baseline. 1.4.