What is a "neural network prior" in this context of physics informed neural networks? In the paper "Physics Informed Deep Learning (Part I): Data-driven solutions of nonlinear partial differential equations" (https://arxiv.org/abs/1711.10561v1), basically this paper uses a neural network to represent the solution of partial differential equations. Training data is the known values on randomly selected dirichlet and neumann boundary points. Interior volume points are also randomly selected, and for these points we can compute a residual that measures how wrong the current NN approximation of the solution is by evaluating the equation for the PDE.
To reduce the number of training points needed in the spatio-temporal domain, for discrete-time problems they use a Runge-Kutta scheme where they compute the solution to one timestep at a time. In RK, whether in implicit or explicit RK schemes, you compute some nodal values first. Then you apply a set of weights to those nodal values to get the extrapolation to the value at the next timestep n+1.
The node points are defined implicitly by (normally this would be solved by solving a linear system of equations, but I'm not sure what they are doing here)
$ u^{n+c_i} = u^n - \Delta t \sum_{j=1}^q a_{ij} N[u^{n+c_j}] $ for i=1, ..., q
Then the extrapolation to the next timestep is
$ u^{n+1} = u^n - \Delta t \sum_{j=1}^q b_j N[u^{n+c_j}] $
As far as I understand, the numbers $c_j$, $b_j$, $a_{ij}$ are supposed to be taken from Butcher tableau for the desired RK model order with $q$ stages. At least, that is what normally happens with normal RK schemes.
Anyway, the paper says "we proceed by placing a multi-output neural network prior on"
$ [u^{n+c_1}(x), \dots, u^{n+c_q}(x), u^{n+1}(x)] $
What is a "neural network prior"? I'm familiar with regular supervised learning from an ML perspective, but I don't know what this means. Is Bayes rule going to be used here? Are they solving for the implicit RK nodal points using a neural network?
Edit:
They are using Butcher tableau that are a few megabytes in size. So, they are not learning the RK numbers $a_{ij}, b_i, c_i$, they are getting them from an external source.
https://github.com/maziarraissi/PINNs/tree/master/Utilities/IRK_weights
Here is one of the places where they load the weights:
https://github.com/maziarraissi/PINNs/blob/master/main/discrete_time_identification%20(KdV)/KdV.py

"Prior" refers to the prior probability distribution of variational Bayesian methods:

*

*https://en.wikipedia.org/wiki/Prior_probability

*https://en.wikipedia.org/wiki/Variational_Bayesian_methods
Certain artificial neural networks are special cases of variational Bayesian methods, such as variational autoencoders (VAEs).
