Transformers, Deep Learning, Time Series Analysis, Forecasting, Machine Learning.
pyramidal attention mechanism that captures hierarchical multi-scale temporal dependencies. Like LogTrans, this model also explicitly encodes a sparsity bias into the attention scheme. FEDFormer [6]: Enhances the traditional transformer architecture by incorporating seasonal-trend decomposition