Convergence of coordinate ascent variational inference for log-concave measures via optimal transport
<|reference_start|>Convergence of coordinate ascent variational inference for log-concave measures via optimal transport: Mean field variational inference (VI) is the problem of finding the closest product (factorized) measure, in the sense of relative entropy, to a given high-dimensional probability measure $\rho$. The well known Coordinate Ascent Variational Inference (CAVI) algorithm aims to approximate this product measure by iteratively optimizing over one coordinate (factor) at a time, which can be done explicitly. Despite its popularity, the convergence of CAVI remains poorly understood. In this paper, we prove the convergence of CAVI for log-concave densities $\rho$. If additionally $\log \rho$ has Lipschitz gradient, we find a linear rate of convergence, and if also $\rho$ is strongly log-concave, we find an exponential rate. Our analysis starts from the observation that mean field VI, while notoriously non-convex in the usual sense, is in fact displacement convex in the sense of optimal transport when $\rho$ is log-concave. This allows us to adapt techniques from the optimization literature on coordinate descent algorithms in Euclidean space.<|reference_end|>
@article{arnese2024convergence,
    title={Convergence of coordinate ascent variational inference for log-concave
  measures via optimal transport},
    author={Manuel Arnese and Daniel Lacker},
    journal={arXiv preprint arXiv:2404.08792},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.08792},
    primaryClass={stat.ML cs.LG math.OC math.PR math.ST stat.TH}
}