We introduce a framework for Continual Learning (CL) based on Bayesian inference over the function space rather than the parameters of a deep neural network. This method, referred to as functional regularisation for Continual Learning, avoids forgetting a previous task by constructing and memorising an approximate posterior belief over the underlying task-specific function. To achieve this we rely on a Gaussian process obtained by treating the weights of the last layer of a neural network as random and Gaussian distributed. Then, the training algorithm sequentially encounters tasks and constructs posterior beliefs over the task-specific functions by using inducing point sparse Gaussian process methods. At each step a new task is first learnt and then a summary is constructed consisting of (i) inducing inputs -a fixed-size subset of the task inputs selected such that it optimally represents the task -and (ii) a posterior distribution over the function values at these inputs. This summary then regularises learning of future tasks, through Kullback-Leibler regularisation terms. Our method thus unites approaches focused on (pseudo-)rehearsal with those derived from a sequential Bayesian inference perspective in a principled way, leading to strong results on accepted benchmarks.
The paper develops a continual learning method based on Gaussian Processes (GPs) applied in the way introduced by prior work as Deep Kernel Learning (DKL). The proposed method summarizes tasks as sparse GPs and use them as regularizers for the subsequent tasks in order to avoid catastrophic forgetting. Salleviating the instability resulting from the representation drift.
Employing inducing point training for task memorization is a novel and interesting idea, which could be useful for the continual learning community. The fact that this approach also captures the uncertainty of the replays contributes fairly to robustness. Lastly, performing knowledge transfer by inheriting the KL term of the ELBO is also interesting, however, its theoretical implications deserve a close look. It would be enlightening to analyze which true posterior the learned model then corresponds to. Would not it be a slightly more principled Bayesian approach (i.e. one that has stronger grounds at first principles) to perform the knowledge transfer to assign the posterior of one task as the prior of the other, alternatively to keeping the entire KL term intact which employs the q(u_i) as the surrogate for q(u_j), i.e. the way introduced by Nguyen et al., 2017?
The presentation clarity of the paper is open for improvement. For instance, the abstract is written in a sort of convoluted way. I do not get how the KL divergence suddenly kicks in and for what exact purpose. Is it variational inference or a hand-designed regularization term?
I find the argumentation from Eq. 1 downwards until the end of Sec 2.1 on BNNs with stochastic weights and their relation to GPs a bit unnecessary complication. These are very well known facts. It would suffice to state briefly that the task learner is a vanilla DKL used within a vanilla sparse GP.
Figure 1 is also not so descriptive. I do not get what the GP here is exactly doing. What is input to and for which output modelity does it find a mapping? What is the calligraphic L in the figure? Is it a neural net loss or an ELBO? 
In general I could not grasp why it makes sense to treat the the output layer params of a neural net treated for continual learning? They will not be sufficient to encode a task anyway, as an expressive enough neural net will leave only a linear mapping to the final layer. What happens if the intermediate representations of the input observations require a domain shift as the tasks evolve?
Overall, the presented ideas are fairly interesting and the experimental results are good enough for proof-of-concept, though not groundbreaking (behind the close relative VCL on MNIST and no comparison against VCL on Omniglot). Hence, this is a decent piece of work that lies somewhere around the borderline. My major concern is that the proposed method is conceptually not novel enough compared to Nguyen et al., 2017. My secondary concern is that the presentation is very much open to improvement in points hinted above.
--
Post-rebuttal: Thanks to authors for their tremendous effort to alleviate my concerns. The fact is that the conceptual novelty of the paper is too slim compared to VCL. As mentioned above, I even find the VCL approach more principled. I could have viewed the outcome of the paper as a slightly bigger news for the community if there was something unexpectedly positive on the reported results. However, as it appears from the comment below, the authors propose a super close variant of VCL that combines a few well-known techniques in a rather straightforward way and achieves in the end a model that performs on par with it. Under these conditions, I have hard time to find a reason to champion this paper for acceptance. That being said, I view this paper a tight borderline case due to its technical depth, hence I will not object to a reject decision either.