Transformers, Deep Learning, Machine Learning, Editors Pick, Data Science.
the task, when we embed our tokens, we may also wish to preserve the ordering of our tokens; this is especially important in domains such as NLP, or we essentially end up with a bag of words approach. To overcome this, we apply positional encoding to our embeddings. Whilst there are multiple ways