STC-ViT: Spatio Temporal Continuous Vision Transformer for Weather Forecasting
<|reference_start|>STC-ViT: Spatio Temporal Continuous Vision Transformer for Weather Forecasting: Operational weather forecasting system relies on computationally expensive physics-based models. Recently, transformer based models have shown remarkable potential in weather forecasting achieving state-of-the-art results. However, transformers are discrete models which limit their ability to learn the continuous spatio-temporal features of the dynamical weather system. We address this issue with STC-ViT, a Spatio-Temporal Continuous Vision Transformer for weather forecasting. STC-ViT incorporates the continuous time Neural ODE layers with multi-head attention mechanism to learn the continuous weather evolution over time. The attention mechanism is encoded as a differentiable function in the transformer architecture to model the complex weather dynamics. We evaluate STC-ViT against a operational Numerical Weather Prediction (NWP) model and several deep learning based weather forecasting models. STC-ViT performs competitively with current data-driven methods in global forecasting while only being trained at lower resolution data and with less compute power.<|reference_end|>
@article{saleem2024stc-vit:,
    title={STC-ViT: Spatio Temporal Continuous Vision Transformer for Weather
  Forecasting},
    author={Hira Saleem, Flora Salim, Cormac Purcell},
    journal={arXiv preprint arXiv:2402.17966},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.17966},
    primaryClass={cs.LG}
}