Simplified Mamba with Disentangled Dependency Encoding for Long-Term Time Series Forecasting
<|reference_start|>Simplified Mamba with Disentangled Dependency Encoding for Long-Term Time Series Forecasting: Recent advances in deep learning have led to the development of numerous models for Long-term Time Series Forecasting (LTSF). However, most approaches still struggle to comprehensively capture reliable and informative dependencies inherent in time series data. In this paper, we identify and formally define three critical dependencies essential for improving forecasting accuracy: the order dependency and semantic dependency in the time dimension as well as cross-variate dependency in the variate dimension. Despite their significance, these dependencies are rarely considered holistically in existing models. Moreover, improper handling of these dependencies can introduce harmful noise that significantly impairs forecasting performance. To address these challenges, we explore the potential of Mamba for LTSF, highlighting its three key advantages to capture three dependencies, respectively. We further empirically observe that nonlinear activation functions used in vanilla Mamba are redundant for semantically sparse time series data. Therefore, we propose SAMBA, a Simplified Mamba with disentangled dependency encoding. Specifically, we first eliminate the nonlinearity of vanilla Mamba to make it more suitable for LTSF. Along this line, we propose a disentangled dependency encoding strategy to endow Mamba with efficient cross-variate dependency modeling capability while minimizing the interference between time and variate dimensions. We also provide rigorous theory as a justification for our design. Extensive experiments on nine real-world datasets demonstrate the effectiveness of SAMBA over state-of-the-art forecasting models.<|reference_end|>
@article{weng2024simplified,
    title={Simplified Mamba with Disentangled Dependency Encoding for Long-Term
  Time Series Forecasting},
    author={Zixuan Weng, Jindong Han, Wenzhao Jiang, Hao Liu},
    journal={arXiv preprint arXiv:2408.12068},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.12068},
    primaryClass={cs.LG}
}