Probabilistic ODE Solvers with Runge-Kutta Means
Runge-Kutta methods are the classic family of solvers for ordinary differential equations (ODEs), and the basis for the state of the art. Like most numerical methods, they return point estimates. We construct a family of probabilistic numerical methods that instead return a Gauss-Markov process defining a probability distribution over the ODE solution. In contrast to prior work, we construct this family such that posterior means match the outputs of the Runge-Kutta family exactly, thus inheriting their proven good properties. Remaining degrees of freedom not identified by the match to Runge-Kutta are chosen such that the posterior probability measure fits the observed structure of the ODE. Our results shed light on the structure of Runge-Kutta solvers from a new direction, provide a richer, probabilistic output, have low computational cost, and raise new research questions.
1. SummaryThis paper presents a probabilistic approach to ODE solvers based on Gaussian processes so that the posterior mean matches exactly the Runge-Kutta estimates. The main contribution of the paper is that it shows that Runge-Kutta solutions of order 1, 2, 3 can be obtained from integrated Weiner processes. 2. QualityThe paper is technically sound as it exploits the closeness property of GPs under linear operators to come up with covariance functions and, consequently, predictive means that match the Runge-Kutta solutions. The claims regarding the matching of these solutions are well supported by the analytical derivations in the paper and in the supplemental material. The claim regarding the low computational cost is also well supported, as it falls directly from the resulting model being a Gauss Markov process. One possible weakness of this work is the claim in the main paper regarding previous ODE solvers that they do not match exactly the Runge-Kutta estimate. This contradicts what is presented in the supplementary material where the authors (line 448 onwards) that Equation (42) in the supplemental material can be used to derive 2nd order and 3rd order Runge-Kutta means. Although such a match is only achieved when \lambda tends to infinity, there should be more discussion in the main paper regarding both approaches. In fact, the authors do recognize (line 414) that the “application of different GP models might still be favorable in practice”, but fall short in providing more discussion or more experimentation comparing the proposed method with previous work.  One additional technical comment is the setting of the “scale” parameter \sigma^2, which seems to be done ad-hoc. Why haven’t  he authors pursued a more principled way to provide point-estimates for this since it ultimately calibrates the the resulting posterior distribution?3. ClarityThe paper is very well written and organized and it contains about the right balance of background and new material. 4. OriginalityThe paper addresses an existing problem, that of providing probabilistic interpretations for ODE solvers. It approaches this problem in a very different way to previous work as it goes beyond simply using GPs and exploiting their closeness under linear operators by showing equivalence results when starting from the kernel for the Wiener process. 5. SignificanceThe results are important as they provide a better understanding of Runge-Kutta methods from a probabilistic point of view. However, the paper will benefit greatly from a more careful comparison wrt to previous probabilistic approaches to the same problem. 6. Minor comments(a) line 268: "Its finite values that this posterior indeed defines a proper GP". I don't think this sentence quite makes sense(b) line 295: "an correction" --> "a correction"