Probabilistic Linear Multistep Methods
We present a derivation and theoretical investigation of the Adams-Bashforth and Adams-Moulton family of linear multistep methods for solving ordinary differential equations, starting from a Gaussian process (GP) framework. In the limit, this formulation coincides with the classical deterministic methods, which have been used as higher-order initial value problem solvers for over a century. Furthermore, the natural probabilistic framework provided by the GP formulation allows us to derive probabilistic versions of these methods, in the spirit of a number of other probabilistic ODE solvers presented in the recent literature [1,2,3,4]. In contrast to higher-order Runge-Kutta methods, which require multiple intermediate function evaluations per step, Adams family methods make use of previous function evaluations, so that increased accuracy arising from a higher-order multistep approach comes at very little additional computational cost. We show that through a careful choice of covariance function for the GP, the posterior mean and standard deviation over the numerical solution can be made to exactly coincide with the value given by the deterministic method and its local truncation error respectively. We provide a rigorous proof of the convergence of these new methods, as well as an empirical investigation (up to fifth order) demonstrating their convergence rates in practice.
%%% UPDATE %%% Thank you for your detailed response.   In discussion, several points were raised by a meta-reviewer about the connection between this work and the work of Schober et al in NIPS 2014. In particular, it was suggested that the two methods may be near identical, albeit with different derivations. I shall not repeat the discussion here as I have only a limited understanding and I do not wish to make statements that are incorrect. I do hope these points will be made available to the author(s) by the meta reviewer(s).  At the outset this paper excited me, although I am not expert in ODEs. It is clearly well written and technically proficient. However, the nature of the concerns that have been raised make it difficult to argue strongly in favour of the paper. As a result, my confidence score and corresponding novelty scores have now significantly decreased from the initial review stage.  I would like to add that it would have been much better if the issue of equivalence with Schober et al 2014 had been put to the authors during the initial review stage, so that there was an opportunity to provide a rebuttal. (A comparison was requested by one reviewer, but at that stage the argument that the two works may be near identical was not put forward.) The unfortunate situation appears to be due to the structure of the NIPS review process and I do sympathise with the author(s)' predicament.  %%% END %%%   This paper develops a probabilistic version of the classical Adams-Bashforth and Adams-Moulton numerical integrators. The authors propose a distribution over (stochastic) solutions of the ordinary differential equation (ODE) that is intended to provide a probabilistic quantification of epistemic uncertainty due to the presence of numerical error. This paper is in line with other recent research into probabilistic numerical methods.
This is an outstanding paper. The author(s) should be warmly congratulated on a timely, intelligent and thought-provoking contribution to this emerging literature.  The only substantive negative comment that I can make is that there is not a frequentist assessment of the coverage properties of the proposed distributional quantification of epistemic uncertainty (empirical or theoretical). However, that would not be easily achieved within the confines of 8 pages.  In general terms, the fundamentalist Bayesian could reject the idea because information is "thrown away" outside the s-length moving window. They could also object because the prior measure on the solution is not placed on an infinite dimensional function space at the outset, but is instead defined on the finite dimensional distributions in a somewhat ad-hoc manner (relative to the fundamentalist Bayesian approach). The paper presumably aims to strike a balance between being "fully" Bayesian and being a computational pragmatist. Perhaps there is room to raise and address this trade-off in discussion?  Some minor comments:  Please provide a reference for the Picard-Lindelof theorem (page 2).  On page 2, lines 55-56, I find the description "marginal measure given evaluations of the vector field" to be rather odd, since it sounds like you are in fact conditioning on evaluations of the vector field. My guess is that the equation that follows this description is not the one that was intended to be written here.  Again on page 2, rather than \mathbb{Z}^+, do you mean \mathbb{N}? As far as I can tell, zero is excluded from this statement, so \mathbb{N} = {1,2,3,...} would be more standard. (I may well be mistaken - perhaps some clarification could be made.)  I wonder how standard it is to refer to (e.g.) Eqn. 10 as a Gaussian "process" prior? Would not simply a "Gaussian prior" suffice here, given there doesn't seem to be a continuous time variable present at this point?  I believe references [2] and [3] are both now published, so these reference should be updated.  In the appendix a bold-face `f' is used, whereas in the main text it is not bold.