Probabilistic Models for Integration Error in the Assessment of Functional Cardiac Models
This paper studies the numerical computation of integrals, representing estimates or predictions, over the output f (x) of a computational model with respect to a distribution p(dx) over uncertain inputs x to the model. For the functional cardiac models that motivate this work, neither f nor p possess a closed-form expression and evaluation of either requires ≈ 100 CPU hours, precluding standard numerical integration methods. Our proposal is to treat integration as an estimation problem, with a joint model for both the a priori unknown function f and the a priori unknown distribution p. The result is a posterior distribution over the integral that explicitly accounts for dual sources of numerical approximation error due to a severely limited computational budget. This construction is applied to account, in a statistically principled manner, for the impact of numerical errors that (at present) are confounding factors in functional cardiac model assessment.
Summary
The paper presents a method for assessing the uncertainty in the evaluation of an expectation over the output of a complex simulation model given uncertainty in the model parameters. Such simulation models take a long time to solve, given a set of parameters, so the task of averaging over the outputs of the simulation given uncertainty in the parameters is challenging. One cannot simply run the model so many times that error in the estimate of the integral is controlled. The authors approach the problem as an inference task. Given samples from the parameter posterior one must infer the posterior over the integral of interest. The parameter posterior can be modelled as a DP mixture and my matching the base measure of the DP mixture with the kernel used in the Gaussian process based Bayesian Quadrature (BQ) procedure for solving the integral. The method is benchmarked using synthetic and a real-world cardiac model against simply using the mean and standard error of simulations from the model for parameter samples. It is shown to be more conservative - the simple standard error approach is over-confident in the case of small sample size.
Comments: 
This is an interesting topic and extends recent developments in probabilistic numerics to the problem of estimating averages over simulations from a model with uncertain parameters. I'm not aware of other principled approaches to solving this problem and I think it is of interest to people at NIPS. 
The paper is well written and easy to follow. A lot of technical detail is in the supplementary so really this is a pretty substantial paper if one considers it all together. Putting technical details in the supplement improves the readability. 
There is a proof of consistency of the approach in a restricted setting which gives some rigorous foundations to the approach. 
The matching of the mixture model components and kernel into conjugate pairs (e.g. Gaussian mixture with squared exponential kernel) seems like a neat trick to make things tractable. 
It is presented as an advantage that the method is conservative, e.g. the broad posterior over the quantity of interest is more likely to contain the ground truth. However, I was not sure what was the reason for the conservative behaviour and how general that behaviour is - is it generally the case that the method will be conservative or could there be cases where the procedure also underestimates the uncertainties of interest? Is this simply an empirical observation based on a restricted set of examples? I guess this is difficult to address theoretically as it is not an asymptotic issue.