Many deep-learning based methods such as Bayesian deep learning (DL) and deep reinforcement learning (RL) have heavily relied on the ability of a model being able to efficiently explore via Bayesian sampling. Particle-optimization sampling (POS) is a recently developed technique to generate high-quality samples from a target distribution by iteratively updating a set of interactive particles, with a representative algorithm the Stein variational gradient descent (SVGD). Though obtaining significant empirical success, the non-asymptotic convergence behavior of SVGD remains unknown. In this paper, we generalize POS to a stochasticity setting by injecting random noise in particle updates, called stochastic particle-optimization sampling (SPOS). Notably, for the first time, we develop non-asymptotic convergence theory for the SPOS framework, characterizing convergence of a sample approximation w.r.t. the number of particles and iterations under both convex-and noncovexenergy-function settings. Interestingly, we provide theoretical understanding of a pitfall of SVGD that can be avoided in the proposed SPOS framework, i.e., particles tend to collapse to a local mode in SVGD under some particular conditions. Our theory is based on the analysis of nonlinear stochastic differential equations, which serves as an extension and a complementary development to the asymptotic convergence theory for SVGD such as (Liu, 2017). With such theoretical guarantees, SPOS can be safely and effectively applied on both Bayesian DL and deep RL tasks. Extensive results demonstrate the effectiveness of our proposed framework.
This paper considers the problem of Bayesian inference using particle optimization sampler. Similarly to SGLD, authors propose Stochastic Particle Optimization Sampler (SPOS), augmenting Stein Variational Gradient Descent (SVGD) with diminishing Gaussian noise, replacing the hard-to-compute term of the Chen et al. (2018) formulation. Various theoretical results are given.
This paper was a pleasant read until I decided to check the proof of Theorem 3. I was not able to understand transitions in some of the steps and certain statements in the proof seem wrong.
Theorem 3:
"Note that $\theta^i_t$ and $\hat \theta^i_t$ are initialized with the same initial distribution µ0 = ν0 and we can also set $\theta^i_0$ to be independent of $\hat \theta^i_0$, we can have $\gamma(0) = 0$. $\gamma(0) = E \|\theta^i_0 - \hat \theta^i_0 \|^2$." - this doesn't seem right to me. Expectation of squared difference of two independent and identically distributed random variables is not 0, assuming expectation is with respect to their joint density.
"Then according to the Gronwall Lemma, we have" - I don't see how the resulting inequality was obtained. When I tried applying Gronwall Lemma, it seems that authors forgot to multiply by $t$ and  $\lambda_1$. Could you please elaborate how exactly Gronwall Lemma was used in this case.
"... some positive constants c1 and c2 independent of (M, d)$ - in the proof authors introduce additional assumption "We can tune the bandwidth of the RBF kernel to make ∇K ≤ H_∇K, which is omitted in the Assumption due to the space limit." First, there is a missing norm, since ∇K is a vector and H_∇K is I believe a scalar constant. Second, c1 = H_∇K + H_F, which both bound norm of d-dimensional vector and hence depend on d. I also suggest that all assumptions are included in the theorem statements, especially since authors have another assumption requiring large bandwidth. Additionally, feasibility of these both assumptions being satisfied should be explored (it seems to me that they can hold together, but it doesn't mean that part of assumptions can be moved to the supplement).
I find using Wasserstein-1 metric misleading in the theorem statement . This is not what authors really bound - from the proof it can be seen that they bound W_1 with W_2 and then with just an expectation of l2 norm. Moreover I don't understand the meaning of this bound. Theorem is concerned with W_1 distance between two atomic measures. What is the expectation over? Note that atom locations are supposed to be fixed for the W_1 to make sense in this context (and the expectation is over the coupling of discrete measures defined by weights of the atoms, not atom locations).
"Note the first bullet indicates U to be a convex function and W to be ... " I think it should be K, not W.
Theorems 3-6 could be lemmas, while there should be a unifying theorem for the bound.
Finally, I think notation should be changed - same letter is used for Wasserstein distance and Wiener process.
Other comments:
Example in Figure 1 is somewhat contrived - clearly gradient based particle sampler will never escape the mode since all modes are disconnected by regions with 0 density. Proposed method on the other hand will eventually jump out due to noise, but it doesn't necessarily mean it produces better posterior estimate. Something more realistic like a mixture of Gaussians, with density bounded away from zero across domain space, will be more informative.
It is not sufficient to report RMSE and test log likelihood for BNNs. One of the key motivating points is posterior uncertainty estimation. Hence important metric, when comparing to other posterior inference techniques, is to show high uncertainty for out of distribution samples and low for training/test data.