  Bayesian deep learning plays an important role especially for its ability
evaluating epistemic uncertainty (EU). Due to computational complexity issues,
approximation methods such as variational inference (VI) have been used in
practice to obtain posterior distributions and their generalization abilities
have been analyzed extensively, for example, by PAC-Bayesian theory; however,
little analysis exists on EU, although many numerical experiments have been
conducted on it. In this study, we analyze the EU of supervised learning in
approximate Bayesian inference by focusing on its excess risk. First, we
theoretically show the novel relations between generalization error and the
widely used EU measurements, such as the variance and mutual information of
predictive distribution, and derive their convergence behaviors. Next, we
clarify how the objective function of VI regularizes the EU. With this
analysis, we propose a new objective function for VI that directly controls the
prediction performance and the EU based on the PAC-Bayesian theory. Numerical
experiments show that our algorithm significantly improves the EU evaluation
over the existing VI methods.

Machine Learning;Machine Learning