title: Autoencoding Variational Inference For Topic Models
abstract: Topic models are one of the most popular methods for learning representations of
  text, but a major challenge is that any change to the topic model requires mathematically
  deriving a new inference algorithm. A promising approach to address
  this problem is autoencoding variational Bayes (AEVB), but it has proven diffi-
  cult to apply to topic models in practice. We present what is to our knowledge the
  first effective AEVB based inference method for latent Dirichlet allocation (LDA),
  which we call Autoencoded Variational Inference For Topic Model (AVITM). This
  model tackles the problems caused for AEVB by the Dirichlet prior and by component
  collapsing. We find that AVITM matches traditional methods in accuracy
  with much better inference time. Indeed, because of the inference network, we
  find that it is unnecessary to pay the computational cost of running variational
  optimization on test data. Because AVITM is black box, it is readily applied
  to new topic models. As a dramatic illustration of this, we present a new topic
  model called ProdLDA, that replaces the mixture model in LDA with a product
  of experts. By changing only one line of code from LDA, we find that ProdLDA
  yields much more interpretable topics, even if LDA is trained via collapsed Gibbs
  sampling.
authors: Akash Srivastava Charles Sutton
hIndex: Charles Sutton 43