Transformers, Deep Learning, Time Series Analysis, Forecasting, Machine Learning.
disassembling existing transformers to analyze whether or not the complex design of the transformer was necessary. They broke the attention layer down into a simple linear layer, then removed auxiliary pieces apart from the embedding mechanisms, and finally reduced the transformer down to only