BeamVQ: Aligning Space-Time Forecasting Model via Self-training on Physics-aware Metrics
<|reference_start|>BeamVQ: Aligning Space-Time Forecasting Model via Self-training on Physics-aware Metrics: Data-driven deep learning has emerged as the new paradigm to model complex physical space-time systems. These data-driven methods learn patterns by optimizing statistical metrics and tend to overlook the adherence to physical laws, unlike traditional model-driven numerical methods. Thus, they often generate predictions that are not physically realistic. On the other hand, by sampling a large amount of high quality predictions from a data-driven model, some predictions will be more physically plausible than the others and closer to what will happen in the future. Based on this observation, we propose \emph{Beam search by Vector Quantization} (BeamVQ) to enhance the physical alignment of data-driven space-time forecasting models. The key of BeamVQ is to train model on self-generated samples filtered with physics-aware metrics. To be flexibly support different backbone architectures, BeamVQ leverages a code bank to transform any encoder-decoder model to the continuous state space into discrete codes. Afterwards, it iteratively employs beam search to sample high-quality sequences, retains those with the highest physics-aware scores, and trains model on the new dataset. Comprehensive experiments show that BeamVQ not only gave an average statistical skill score boost for more than 32% for ten backbones on five datasets, but also significantly enhances physics-aware metrics.<|reference_end|>
@article{wu2024beamvq:,
    title={BeamVQ: Aligning Space-Time Forecasting Model via Self-training on
  Physics-aware Metrics},
    author={Hao Wu, Xingjian Shi, Ziyue Huang, Penghao Zhao, Wei Xiong, Jinbao
  Xue, Yangyu Tao, Xiaomeng Huang, Weiyan Wang},
    journal={arXiv preprint arXiv:2405.17051},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.17051},
    primaryClass={cs.LG cs.AI}
}