The contrastive divergence algorithm is a popular approach to training energybased latent variable models, which has been widely used in many machine learning models such as the restricted Boltzmann machines and deep belief nets. Despite its empirical success, the contrastive divergence algorithm is also known to have biases that severely affect its convergence. In this article we propose an unbiased version of the contrastive divergence algorithm that completely removes its bias in stochastic gradient methods, based on recent advances on unbiased Markov chain Monte Carlo methods. Rigorous theoretical analysis is developed to justify the proposed algorithm, and numerical experiments show that it significantly improves the existing method. Our findings suggest that the unbiased contrastive divergence algorithm is a promising approach to training general energy-based latent variable models.
The paper proposes an algorithmic improvement that significantly simplifies training of energy-based models, such as the Restricted Boltzmann Machine. The key issue in training such models is computing the gradient of the log partition function, which can be framed as computing the expected value of f(x) = dE(x; theta) / d theta over the model distribution p(x). The canonical algorithm for this problem is Contrastive Divergence which approximates x ~ p(x) with k steps of Gibbs sampling, resulting in biased gradients. In this paper, the authors apply the recently introduced unbiased MCMC framework of Jacob et al. to completely remove the bias. The key idea is to (1) rewrite the expectation as a limit of a telescopic sum: E f(x_0) + \sum_t E f(x_t) - E f(x_{t-1}); (2) run two coupled MCMC chains, one for the “positive” part of the telescopic sum and one for the “negative” part until they converge. After convergence, all remaining terms of the sum are zero and we can stop iterating. However, the number of time steps until convergence is now random.
Other contributions of the paper are:
1. Proof that Bernoulli RBMs and other models satisfying certain conditions have finite expected number of steps and finite variance of the unbiased gradient estimator.
2. A shared random variables method for the coupled Gibbs chains that should result in faster convergence of the chains.
3. Verification of the proposed method on two synthetic datasets and a subset of MNIST, demonstrating more stable training compared to contrastive divergence and persistent contrastive divergence.
I am very excited about this paper and strongly support its acceptance, since the proposed method should revitalize research in energy-based models. While I find the experiments to be somewhat lacking, this is sufficiently offset by the theoretical contributions of the paper.
Pros
1. The paper reads well and introduces all the necessary preliminaries to understand the method. This is important, since I expect many readers to be unfamiliar with the technique.
2. The proposed method solves an important problem which, as far as I understand, has been the roadblock in large-scale training of RBMs and related models. It is also elegant and fairly straightforward to implement.
3. The proof of finite computation time and variance is very nice to have. This is because in some cases removing the bias leads to infinite variance, e.g. a parallel submission on SUMO (https://openreview.net/forum?id=SylkYeHtwr).
Cons
1. I don’t think Corollary 1 (convergence of gradient descent to the global optimum) is true for RBMs, as stated on Page 6. This is because the log-likelihood of RBM, or indeed any latent-variable model with permutation-invariant latents, is non-convex. I would suggest removing this corollary and simplifying Algorithm 2 to be regular SGD, as used in the experiments.
2. There is no experimental comparison of Algorithm 1 (the general version) and Algorithm 3 (the specialized RBM version). It seems intuitive that the specialized version should have lower computation time, but this must be confirmed.
3. The experimental section may be significantly improved.
* It is unclear what value of k (number of initial Gibbs steps) from Algorithm 2 is used.
* The experiments on just the “0” digits of MNIST seem a bit simplistic for the year 2019. It is also not clear what binarization protocol is used.
* It would be very helpful to provide estimates of the gradient (not log-likelihood) variance of each method to better understand the trade-off between the bias and the variance.
* I would also like to see the wall-clock time comparison of the methods.
Minor comments
* Page 1. Of this kind -> of this class. The data distribution p_v (v; theta) -> The model distribution
* Page 2. Property -> properties. CD-\tau -- I don’t think you can correctly refer to your method in this way, since it has at least double the computation time of CD for the same number of iterations.
* Page 3. Provides -> provide. Likelihood gradient -> log-likelihood gradient
* Algorithm 1 is an infinite loop with no break clause. It would be good to add a break statement after line 5. This would also simplify the discussion of the method.
* Page 7. I wouldn’t call the fact that CD doesn’t converge on the BAS dataset remarkable, given that it’s been reported by Fischer & Igel 2014.
* Page 9. The last paragraph stating that the proposed method is not a replacement for CD is confusing. Can you add a short experiment to demonstrate that this combination makes sense?