Expectation Particle Belief Propagation
We propose an original particle-based implementation of the Loopy Belief Propagation (LPB) algorithm for pairwise Markov Random Fields (MRF) on a continuous state space. The algorithm constructs adaptively efficient proposal distributions approximating the local beliefs at each note of the MRF. This is achieved by considering proposal distributions in the exponential family whose parameters are updated iterately in an Expectation Propagation (EP) framework. The proposed particle scheme provides consistent estimation of the LBP marginals as the number of particles increases. We demonstrate that it provides more accurate results than the Particle Belief Propagation (PBP) algorithm of [1] at a fraction of the computational cost and is additionally more robust empirically. The computational complexity of our algorithm at each iteration is quadratic in the number of particles. We also propose an accelerated implementation with sub-quadratic computational complexity which still provides consistent estimates of the loopy BP marginal distributions and performs almost as well as the original procedure.
The authors propose a variation on particle belief propagation in which the proposal distribution is adaptively selected at each step of the algorithm by using an expectation-propagation like calculation: the particle messages are mapped to an exponential family distribution (Gaussian for the experiments).The idea is simple but appealing.The authors show that it is more efficient than a previously proposed MCMC procedure for resampling the particles (whose inner loop is comparatively slow and requires user selection or tuning), and a "fixed" proposal based on EP which is not updated during the particle message passing.The experiments are sufficient to support the claims but could have been stronger.The authors assess the convergence behavior of the algorithm in very simple models (few variables; mildly multimodal or skewed potentials; amenable to simple discretization for comparison), with a simple synthetic denoising example as well.While sufficient to demonstrate the point of the paper, that the EP-like proposals improve convergence in the number of samples, a practical application or example would have strengthened the submission.The authors also mention a practical speed-up based on Briers et al. (2010), which uses subsampling to reduce the computational overhead.While potentially useful, I did not see a significant contribution here.Also, its usefulness and appropriateness will likely depend on the model and number of samples being used.Given that fact, the simple empirical assessment was not strongly compelling. Overall, the paper provides a simple but appealing idea for designing proposals during particle BP.However, it could also benefit from a stronger and more detailed experimental evaluation, in more compelling or practical domains.