Towards Data Science, Deep Learning, Machine Learning, Neural Networks, Naturallanguageprocessing.
of embeddings, one embedding per position, where each position embedding contains not only the embedding of the original word at the position but also information about other words, that it learned using attention. This is then fed into the decoder part of the transformer network, as seen in figure