Enhancing Uncertainty Estimation and Interpretability with Bayesian Non-negative Decision Layer
Xinyue Hu Zhibin Duan Bo Chen Mingyuan Zhou
Although deep neural networks have demonstrated significant success due to their
powerful expressiveness, most models struggle to meet practical requirements for
uncertainty estimation. Concurrently, the entangled nature of deep neural net-
works leads to a multifaceted problem, where various localized explanation tech-
niques reveal that multiple unrelated features influence the decisions, thereby un-
dermining interpretability. To address these challenges, we develop a Bayesian
Nonnegative Decision Layer (BNDL), which reformulates deep neural networks
as a conditional Bayesian non-negative factor analysis. By leveraging stochastic
latent variables, the BNDL can model complex dependencies and provide robust
uncertainty estimation. Moreover, the sparsity and non-negativity of the latent
variables encourage the model to learn disentangled representations and decision
layers, thereby improving interpretability. We also offer theoretical guarantees
that BNDL can achieve effective disentangled learning. In addition, we developed
a corresponding variational inference method utilizing a Weibull variational in-
ference network to approximate the posterior distribution of the latent variables.
Our experimental results demonstrate that with enhanced disentanglement capa-
bilities, BNDL not only improves the model’s accuracy but also provides reliable
uncertainty estimation and improved interpretability.
Factor Analysis, Uncertainty Estimation, explainable AI, Bayesian Last Layer