Sequential-in-time training of nonlinear parametrizations for solving time-dependent partial differential equations
<|reference_start|>Sequential-in-time training of nonlinear parametrizations for solving time-dependent partial differential equations: Sequential-in-time methods solve a sequence of training problems to fit nonlinear parametrizations such as neural networks to approximate solution trajectories of partial differential equations over time. This work shows that sequential-in-time training methods can be understood broadly as either optimize-then-discretize (OtD) or discretize-then-optimize (DtO) schemes, which are well known concepts in numerical analysis. The unifying perspective leads to novel stability and a posteriori error analysis results that provide insights into theoretical and numerical aspects that are inherent to either OtD or DtO schemes such as the tangent space collapse phenomenon, which is a form of over-fitting. Additionally, the unified perspective facilitates establishing connections between variants of sequential-in-time training methods, which is demonstrated by identifying natural gradient descent methods on energy functionals as OtD schemes applied to the corresponding gradient flows.<|reference_end|>
@article{zhang2024sequential-in-time,
    title={Sequential-in-time training of nonlinear parametrizations for solving
  time-dependent partial differential equations},
    author={Huan Zhang and Yifan Chen and Eric Vanden-Eijnden and Benjamin
  Peherstorfer},
    journal={arXiv preprint arXiv:2404.01145},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.01145},
    primaryClass={math.NA cs.LG cs.NA}
}