Towards Data Science, Neural Networks, Deep Learning, Machine Learning, Naturallanguageprocessing.
(h). That is there can be h number of linear layers giving h attention outputs which are then concatenated together. And this is exactly why it is called multi-head attention (multiple heads). The simpler version of figure 11, but with h number of heads is shown in figure 12. Figure 12. Multi-Head