The Case for Bayesian Deep Learning
I'm a self-described Bayesian* at my day job, but the author needs to do better to convince me that the Bayesian approach is worth it in the deep learning space. As far as I can tell, deep learning folks don't give two shits about uncertainty intervals, much less marginalization. All that matters is minimizing that test error as fast as possible. So what if you get a posterior for each parameter... Who cares about the parameters in a neural network as long as the predictions seem well calibrated?The most convincing rationale for adopting a Bayesian perspective is contained the collected works of Jim Berger, which I see is cited by the author... but not used in the manuscript.* Of course, a Bayesian is just a statistician that uses Bayesian techniques even when it's not appropriate -- Andrew Gelman