VISA: Variational Inference with Sequential Sample-Average Approximations
<|reference_start|>VISA: Variational Inference with Sequential Sample-Average Approximations: We present variational inference with sequential sample-average approximation (VISA), a method for approximate inference in computationally intensive models, such as those based on numerical simulations. VISA extends importance-weighted forward-KL variational inference by employing a sequence of sample-average approximations, which are considered valid inside a trust region. This makes it possible to reuse model evaluations across multiple gradient steps, thereby reducing computational cost. We perform experiments on high-dimensional Gaussians, Lotka-Volterra dynamics, and a Pickover attractor, which demonstrate that VISA can achieve comparable approximation accuracy to standard importance-weighted forward-KL variational inference with computational savings of a factor two or more for conservatively chosen learning rates.<|reference_end|>
@article{zimmermann2024visa:,
    title={VISA: Variational Inference with Sequential Sample-Average
  Approximations},
    author={Heiko Zimmermann, Christian A. Naesseth, Jan-Willem van de Meent},
    journal={arXiv preprint arXiv:2403.09429},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.09429},
    primaryClass={stat.ML cs.LG}
}