Nonparametric Bayesian inference on multivariate exponential families
We develop a model by choosing the maximum entropy distribution from the set of models satisfying certain smoothness and independence criteria; we show that inference on this model generalizes local kernel estimation to the context of Bayesian inference on stochastic processes. Our model enables Bayesian inference in contexts when standard techniques like Gaussian process inference are too expensive to apply. Exact inference on our model is possible for any likelihood function from the exponential family. Inference is then highly efficient, requiring only O (log N ) time and O (N ) space at run time. We demonstrate our algorithm on several problems and show quantifiable improvement in both speed and performance relative to models based on the Gaussian process.
Brief Discussion:The paper presents a method for generalisation of local kernel methods to the Bayesian stochastic process setting. In particular they define a smoothness condition that allows them to obtain under certain assumptions a link between a Guassian process kernel and the smoothness condition imposed on the likelihood.QualityThe paper is well written but would benefit from more detailed discussion on the relationship between the kernel and the function rho(.,.) for smoothness. In particular some examples and discussion on whether this is always possible to achieve for any class of kernel would be beneficial. This could be done directly after Theorem 1. In addition, the results presented on the extended likelihood could be more carefully explained and discussed.The authors should further motivate the assumption of independence which does hold in general settings - what are the settings this is reasonable in practice, what are the implications of this assumption on the inference otherwise in practical examples proposed. Explain further how it relates to the other methods cited in this section on sparsification - what exactly is the relationships perhaps make a small illustrative example of the link. In addition the discussion on the equivalence of the interaction between input variables either in the likelihood function or in the prior is interesting and should be expanded. In practical settings which would you choose in general to model the interaction in the likelihood or the prior, comment on this remark, and is this equivalence always true - under what conditions? please comment.Clarity - the paper is generally well written and conciseOriginality - the paper is interesting though the contributions are incremental in natureSignficance - the paper is interesting for the audience.