A Bayesian Perspective on Generalization and Stochastic Gradient Descent
This result is an amazing twofer. Not only do they give a formal probabilistic perspective on how deep nets fall into local minima, the result proposed a way to reduce two "voodoo parameters" - learning rate and batch size - to one, by showing how to pick the other in a way that is optimal for expected generalization.