title: Mixture Distributions for Scalable Bayesian Inference
abstract: Bayesian Neural Networks (BNNs) provides a mathematically grounded framework to quantify uncertainty. However BNNs are computationally inefficient,
  thus are generally not employed on complicated machine learning tasks. Deep
  Ensembles were introduced as a Bootstrap inspired frequentist approach to the
  community, as an alternative to BNN’s. Ensembles of deterministic and stochastic networks are a good uncertainty estimator in various applications (Although,
  they are criticized for not being Bayesian). We show Ensembles of deterministic
  and stochastic Neural Networks can indeed be cast as an approximate Bayesian
  inference. Deep Ensembles have another weakness of having high space complexity, we provide an alternative to it by modifying the original Bayes by Backprop (BBB) algorithm to learn more general concrete mixture distributions over
  weights. We show our methods and its variants can give better uncertainty estimates at a significantly lower parametric overhead than Deep Ensembles. We
  validate our hypothesis through experiments like non-linear regression, predictive
  uncertainty estimation, detecting adversarial images and exploration-exploitation
  trade-off in reinforcement learning.
authors: Pranav Poduval Hrushikesh Loya Rajat Patel Sumit Jain
hIndex: Rajat Patel 80