Towards Data Science, Deep Learning, Machine Learning, Neural Networks, Naturallanguageprocessing.
the position of the words in a sentence. This helps the transformer to understand that one word comes after the other in a sentence. This is where positional embeddings come in handy. This is sort of a vector encoding defining the position of a word. This positional embedding is added to the input