Building Flexible Machine Learning Models for Scientific Computing at Scale
<|reference_start|>Building Flexible Machine Learning Models for Scientific Computing at Scale: Foundation models have revolutionized knowledge acquisition across domains, and our study introduces OmniArch, a paradigm-shifting approach designed for building foundation models in multi-physics scientific computing. OmniArch's pre-training involves a versatile pipeline that processes multi-physics spatio-temporal data, casting forward problem learning into scalable auto-regressive tasks, while our novel Physics-Informed Reinforcement Learning (PIRL) technique during fine-tuning ensures alignment with physical laws. Pre-trained on the comprehensive PDEBench dataset, OmniArch not only sets new performance benchmarks for 1D, 2D and 3D PDEs but also demonstrates exceptional adaptability to new physics via few-shot and zero-shot learning approaches. The model's representations further extend to inverse problem-solving, highlighting the transformative potential of AI-enabled Scientific Computing(AI4SC) foundation models for engineering applications and physics discovery.<|reference_end|>
@article{chen2024building,
    title={Building Flexible Machine Learning Models for Scientific Computing at
  Scale},
    author={Tianyu Chen, Haoyi Zhou, Ying Li, Hao Wang, Chonghan Gao, Rongye Shi,
  Shanghang Zhang, Jianxin Li},
    journal={arXiv preprint arXiv:2402.16014},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.16014},
    primaryClass={cs.LG cs.AI}
}