sTransformer: A Modular Approach for Extracting Inter-Sequential and Temporal Information for Time-Series Forecasting
<|reference_start|>sTransformer: A Modular Approach for Extracting Inter-Sequential and Temporal Information for Time-Series Forecasting: In recent years, numerous Transformer-based models have been applied to long-term time-series forecasting (LTSF) tasks. However, recent studies with linear models have questioned their effectiveness, demonstrating that simple linear layers can outperform sophisticated Transformer-based models. In this work, we review and categorize existing Transformer-based models into two main types: (1) modifications to the model structure and (2) modifications to the input data. The former offers scalability but falls short in capturing inter-sequential information, while the latter preprocesses time-series data but is challenging to use as a scalable module. We propose $\textbf{sTransformer}$, which introduces the Sequence and Temporal Convolutional Network (STCN) to fully capture both sequential and temporal information. Additionally, we introduce a Sequence-guided Mask Attention mechanism to capture global feature information. Our approach ensures the capture of inter-sequential information while maintaining module scalability. We compare our model with linear models and existing forecasting models on long-term time-series forecasting, achieving new state-of-the-art results. We also conducted experiments on other time-series tasks, achieving strong performance. These demonstrate that Transformer-based structures remain effective and our model can serve as a viable baseline for time-series tasks.<|reference_end|>
@article{yin2024stransformer:,
    title={sTransformer: A Modular Approach for Extracting Inter-Sequential and
  Temporal Information for Time-Series Forecasting},
    author={Jiaheng Yin, Zhengxin Shi, Jianshen Zhang, Xiaomin Lin, Yulin Huang,
  Yongzhi Qi, Wei Qi},
    journal={arXiv preprint arXiv:2408.09723},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.09723},
    primaryClass={cs.LG}
}