Transformers, Pytorch, Machine Learning, Deep Dives, Data Science.
receiving and treating an input, a transformer has no sense of order as it looks at the sequence as a whole, in opposition to what RNNs do. We therefore need to add a hint of temporal order so that the transformer can learn dependencies. The specific details of how positional encoding works is out