Transformers, Deep Learning, Time Series Analysis, Forecasting, Machine Learning.
self-attention is permutation-invariant (ignores order) and time series are permutation-sensitive, positional encoding and self-attention will not be enough to capture temporal information. To test this, the authors modified the sequences by shuffling the data and exchanging the first half of the