Online Structured Laplace Approximations for Overcoming Catastrophic Forgetting
We introduce the Kronecker factored online Laplace approximation for overcoming catastrophic forgetting in neural networks. The method is grounded in a Bayesian online learning framework, where we recursively approximate the posterior after every task with a Gaussian, leading to a quadratic penalty on changes to the weights. The Laplace approximation requires calculating the Hessian around a mode, which is typically intractable for modern architectures. In order to make our method scalable, we leverage recent block-diagonal Kronecker factored approximations to the curvature. Our algorithm achieves over 90% test accuracy across a sequence of 50 instantiations of the permuted MNIST dataset, substantially outperforming related methods for overcoming catastrophic forgetting.
*** Edit after the rebuttal period: I have read other reviews and the author feedback. The experiments in this paper are solid and the storyline is clear, however, the proposed method is of arguably low novelty. I will keep my evaluation and rating as is. *** Summary: The paper considers online structured Laplace approximation for training neural networks to mitigate catastrophic forgetting. The idea is to perform Laplace approximation for a task to obtain a Gaussian approximation to the posterior, use this Gaussian as the prior for the weights for next task, and repeat. This results in an addition of an L2 regularisation term to the objective and requires the Hessian of the objective evaluated at the MAP estimate — this is approximated by a Kronecker factored empirical Fisher information matrix. Adding multiple penalties as in EWC and reweighting the empirical FIM term were considered. The take-home message seems to be that using a non-diagonal approximation to the posterior gives a significant gain for all methods across many supervised training tasks.  Comments:  The paper is very well written and in my opinion deserves acceptance. The key idea is relatively simple in hindsight, that is to use structured curvature information for the Laplace approximation (Ritter et al, 2018) in the continual learning setting. Previous works such as EWC, which can be interpreted as online Laplace approximation, only consider a diagonal approximation to the Fisher information matrix. The experiments showed that the addition of the structured approximation is clearly beneficial to online learning, in a similar fashion to the same structured approximation yielding better Laplace approximation in the batch setting (Ritter et al, 2018).  This paper combines several existing techniques and approximations to form a principled approach to continual learning. One could argue that the novelty of this paper is incremental, but the practical advantage of the proposed method seems significant and, as such, could generate interests in the community as continual learning has been an active area of research in the last couple of years.  One thing I dislike is that this method relies on a validation set to tune the regularisation hyper-parameters and the performance is very sensitive to these hyperparams, and it feels unsatisfying that vanilla (approximate) online Bayesian updating is not sufficient to warrant good performance. The variational continual learning framework of Nguyen et al (2018) for example does not introduce any additional hyperparameters to reweight the regularisation term and the expected likelihood term, while still performing quite well with a diagonal Gaussian approximation.