Transformer, Machine Translation, NLP, Attention Mechanism.
transformation, these are Q for Query, K for Key, V for Value. Scalar dot product is done between Q and K and softmax function is applied applied, which is multiplied with V to get the Self Attention value which represents context vector for the word. In Multi-head Self-Attention, word embedding is