STGformer: Efficient Spatiotemporal Graph Transformer for Traffic Forecasting
<|reference_start|>STGformer: Efficient Spatiotemporal Graph Transformer for Traffic Forecasting: Traffic forecasting is a cornerstone of smart city management, enabling efficient resource allocation and transportation planning. Deep learning, with its ability to capture complex nonlinear patterns in spatiotemporal (ST) data, has emerged as a powerful tool for traffic forecasting. While graph neural networks (GCNs) and transformer-based models have shown promise, their computational demands often hinder their application to real-world road networks, particularly those with large-scale spatiotemporal interactions. To address these challenges, we propose a novel spatiotemporal graph transformer (STGformer) architecture. STGformer effectively balances the strengths of GCNs and Transformers, enabling efficient modeling of both global and local traffic patterns while maintaining a manageable computational footprint. Unlike traditional approaches that require multiple attention layers, STG attention block captures high-order spatiotemporal interactions in a single layer, significantly reducing computational cost. In particular, STGformer achieves a 100x speedup and a 99.8\% reduction in GPU memory usage compared to STAEformer during batch inference on a California road graph with 8,600 sensors. We evaluate STGformer on the LargeST benchmark and demonstrate its superiority over state-of-the-art Transformer-based methods such as PDFormer and STAEformer, which underline STGformer's potential to revolutionize traffic forecasting by overcoming the computational and memory limitations of existing approaches, making it a promising foundation for future spatiotemporal modeling tasks.<|reference_end|>
@article{wang2024stgformer:,
    title={STGformer: Efficient Spatiotemporal Graph Transformer for Traffic
  Forecasting},
    author={Hongjun Wang, Jiyuan Chen, Tong Pan, Zheng Dong, Lingyu Zhang, Renhe
  Jiang, and Xuan Song},
    journal={arXiv preprint arXiv:2410.00385},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.00385},
    primaryClass={cs.LG cs.AI cs.DB}
}