Variational Inference with Tail-adaptive f -Divergence
Variational inference with α-divergences has been widely used in modern probabilistic machine learning. Compared to Kullback-Leibler (KL) divergence, a major advantage of using α-divergences (with positive α values) is their mass-covering property. However, estimating and optimizing α-divergences require to use importance sampling, which may have large or infinite variance due to heavy tails of importance weights. In this paper, we propose a new class of tail-adaptive fdivergences that adaptively change the convex function f with the tail distribution of the importance weights, in a way that theoretically guarantees finite moments, while simultaneously achieving mass-covering properties. We test our method on Bayesian neural networks, and apply it to improve a recent soft actor-critic (SAC) algorithm (Haarnoja et al., 2018) in deep reinforcement learning. Our results show that our approach yields significant advantages compared with existing methods based on classical KL and α-divergences.
# Paper ID 2274  Variational Inference with Tail Adapted f-divergence   - The paper proposes an improved variational inference method, f-divergence method that improves over alpha divergence (and KL divergence). The paper is well written and the problem considered is extremely relevant to NIPS. The ideas are theoretically justified and the approximations made are principled. The experiments are well designed and show good performance compared to alpha divergence.  - The method seeks to improve upon alpha divergence. Specifically, the alpha parameter, kept constant during trianing, may experience problems associated with fat-tailed distributions due to variation in the density ratio. Consequently, adjusting alpha is difficult. A constant alpha is the current state-of-the-art which this algorithm improves upon.  - The main contribution is a theoretically justified method to compute the gradient of the f divergence without knowing f itself. Both variants (score function, reparameterization) are described. After making computational approximations, the resulting algorithm is a stochastic approximation to the gradient of the f-divergence. Computationally, the key simplification to the approximation is using the (non-negative) rank of the density ratio to weight each data point in the sample.  - The experimental section is compelling. The results demonstrate the practical utility of the ideas and approximations. In particular, the method outperforms the previous state-of-the-art (alpha divergence) on the hardest instances. This aligns nicely with the theory making for a very nice result. The same trend occurs in all three problem types considered (mixture model, regression task, SAC).  - As far as I can tell, this paper proposes a novel variational inference algorithm with strong theoretical justifications, improving upon the previous best method. The experiments clearly show the superiority of the proposed method. The paper is very well written (with a few minor typos) and seems like an novel contribution to the variational inference literature.