  Physics-informed neural networks (PINNs) incorporate physical knowledge from
the problem domain as a soft constraint on the loss function, but recent work
has shown that this can lead to optimization difficulties. Here, we study the
impact of the location of the collocation points on the trainability of these
models. We find that the vanilla PINN performance can be significantly boosted
by adapting the location of the collocation points as training proceeds.
Specifically, we propose a novel adaptive collocation scheme which
progressively allocates more collocation points (without increasing their
number) to areas where the model is making higher errors (based on the gradient
of the loss function in the domain). This, coupled with a judicious restarting
of the training during any optimization stalls (by simply resampling the
collocation points in order to adjust the loss landscape) leads to better
estimates for the prediction error. We present results for several problems,
including a 2D Poisson and diffusion-advection system with different forcing
functions. We find that training vanilla PINNs for these problems can result in
up to 70% prediction error in the solution, especially in the regime of low
collocation points. In contrast, our adaptive schemes can achieve up to an
order of magnitude smaller error, with similar computational complexity as the
baseline. Furthermore, we find that the adaptive methods consistently perform
on-par or slightly better than vanilla PINN method, even for large collocation
point regimes. The code for all the experiments has been open sourced.

Machine Learning;Computational Physics