Transformers, Deep Learning, Machine Learning, Editors Pick, Data Science.
Scaled dot-product Self Attention: assuming embeddings are positionally encoded From the image, we can see that the linear projections are labelled Q, K and V. In the original paper, these projections were named Query, Key and Value, supposedly taking inspiration from information retrieval.