Towards Data Science, Neural Networks, Deep Learning, Machine Learning, Naturallanguageprocessing.
give one final attention output. This representation is shown in figure 11. Figure 11. Multi-head attention with 3 linear layers (Source: image created by author) But 3 is just a random number we chose. In the actual scenario, these can be any number of linear layers, and these are called heads