ANaGRAM: A Natural Gradient Relative to Adapted Model for efficient PINNs learning
Nilo Schwencke Cyril Furtlehner
In the recent years, Physics Informed Neural Networks (PINNs) have received strong interest as a method to solve PDE driven systems, in particular for data assimilation purpose. This method is still in its infancy, with many shortcomings and failures that remain not properly understood.
In this paper we propose a natural gradient approach to PINNs which contributes to speed-up and improve the accuracy of the training.
Based on an  in depth analysis of the differential geometric structures of the problem, we come up with two distinct contributions:
(i) a new natural gradient algorithm that scales as $\min(P^2S, S^2P)$, where $P$ is the number of parameters, and $S$ the batch size;
(ii) a mathematically principled reformulation of the PINNs problem that allows the extension of natural gradient to it, with proved connections to Green's function theory.
PINNs, SciML, PDEs, Natural Gradient, Neural Tangent Kernel
We propose a new natural gradient that scales as $\min(P^2S, S^2P)$, $P$ the number of parameters and $S$ the batch size, focusing on its application to PINNs traning.