Towards Data Science, Deep Learning, Machine Learning, Neural Networks, Naturallanguageprocessing.
is exactly the same as self-attention with an addition of two blocks (Scale and Mask). To know in detail about Sef-Attention please go to this link — 2.1 Self-Attention. Scaled Attention, as seen in figure 6, is exactly the same, except it adds a scale after the first matrix multiplication