Towards Data Science, Deep Learning, Machine Learning, Neural Networks, Naturallanguageprocessing.
as a ‘query’ and finds some ‘keys’ that correspond to some other words in the sentence, and then take a convex combination of the corresponding ‘values’. In multi-head attention, multiple values, queries, and keys are chosen which provides multiple attention (better word embedding with context).