TENG: Time-Evolving Natural Gradient for Solving PDEs With Deep Neural Nets Toward Machine Precision
<|reference_start|>TENG: Time-Evolving Natural Gradient for Solving PDEs With Deep Neural Nets Toward Machine Precision: Partial differential equations (PDEs) are instrumental for modeling dynamical systems in science and engineering. The advent of neural networks has initiated a significant shift in tackling these complexities though challenges in accuracy persist, especially for initial value problems. In this paper, we introduce the $\textit{Time-Evolving Natural Gradient (TENG)}$, generalizing time-dependent variational principles and optimization-based time integration, leveraging natural gradient optimization to obtain high accuracy in neural-network-based PDE solutions. Our comprehensive development includes algorithms like TENG-Euler and its high-order variants, such as TENG-Heun, tailored for enhanced precision and efficiency. TENG's effectiveness is further validated through its performance, surpassing current leading methods and achieving $\textit{machine precision}$ in step-by-step optimizations across a spectrum of PDEs, including the heat equation, Allen-Cahn equation, and Burgers' equation.<|reference_end|>
@article{chen2024teng:,
    title={TENG: Time-Evolving Natural Gradient for Solving PDEs With Deep Neural
  Nets Toward Machine Precision},
    author={Zhuo Chen, Jacob McCarran, Esteban Vizcaino, Marin Soljav{c}i'c, Di
  Luo},
    journal={arXiv preprint arXiv:2404.10771},
    year={2024},
    number={MIT-CTP/5706},
    archivePrefix={arXiv},
    eprint={2404.10771},
    primaryClass={cs.LG cs.NA math.NA physics.comp-ph}
}