This work addresses continual learning for non-stationary data, using Bayesian neural networks and memory-based online variational Bayes. We represent the posterior approximation of the network weights by a diagonal Gaussian distribution and a complementary memory of raw data. This raw data corresponds to likelihood terms that cannot be well approximated by the Gaussian. We introduce a novel method for sequentially updating both components of the posterior approximation. Furthermore, we propose Bayesian forgetting and a Gaussian diffusion process for adapting to non-stationary data. The experimental results show that our update method improves on existing approaches for streaming data. Additionally, the adaptation methods lead to better predictive performance for non-stationary data.
Contributions:
The paper extends variational continual learning with memory. In this setting the posterior distribution of the model parameters is approximated using a mean-field Gaussian approximation and a small set of datapoints is kept in a memory to combat catastrophic forgetting.
- The paper proposes a new rule for updating the memory and the Gaussian approximation after examining each batch of points. For the memory update, instead of the previously used k-means or random sampling, it examines the factor 'r(d|w)' that each datapoint contributes and selects the most significant ones.
- A second contribution is the use of two adaptation methods in the online learning setting in case there is a distribution shift in the data: Bayesian forgetting and the Ornstein-Uhlenbeck process.
(1) The first contribution, the new memory update rule and the Gaussian update, is novel to my knowledge. The idea is to calculate the Gaussian factor that each datapoint contributes to the posterior (the update corresponding to each datapoint in the message-passing interpretation) and the select the points that contribute the most. Effectively, it calculates the change in the ELBO if each candidate datapoint was moved to the memory and selects the set that minimizes the ELBO of the remaining points. This  is a sensible heuristic because we want to keep datapoints in the memory that contribute the most to the posterior approximation. If a datapoint contributes nothing to the posterior then it shouldn't be kept in memory.
The idea clever and it seems to work well in practice. Questions:
- The Gaussian that each datapoint contributes is an approximation in the sense that if we take that datapoint and its contribution away, the remaining approximate posterior might be suboptimal. This has the consequence that S_tk (eq. 8) is only an approximation to what the optimal ELBO would be without the datapoints selected for the memory. Did the authors conduct experiments or have thoughts on how well S_tk approximates the ELBO of the variational approximation of the remaining points?
-Regarding the experiments, there is a comparison to k-centre and random selection, both of which are proposed in VCL. I am very surprised to see in the experiments that both of these are outperformed by 'no-memory'. For example on 'energy', the baselines at t50 are outperformed by the non memory model at t0. Can this all be contributed to the baselines training their variational approximation without the memory at t0?
- It is unclear whether the gains can be contributed to the new Gaussian update rule or the new memory update rule. To see the contribution of each, there should be an experiment where the proposed Gaussian update rule is used along with k-centre for the memory update. The fact the the no-memory model performed to close to GRS suggests that k-centre with the new Gaussian update rule would be even closer to it.
(2) The second contribution of the paper is the use of Bayesian forgetting and OU to use to deal with the data distribution shift. It shows how these two approaches can be used along with the proposed Gaussian and memory updates. The paper is already very long, so there likely isn't any space for fleshing out this section, but it would be nice to have the experimental results included in the main paper, because most of the experiments are left to the appendix. Perhaps it would be a good idea to focus only on Bayesian forgetting (or the OU process) and try to shorten the section a bit.
In terms of writing and clarity, I have no complaints. The paper is well written and easy to understand.
Minor:
- 3.2 'If the likelihood term p(dtk|w) is well approximated by r(w;dtk)' -  I find this sentence a bit confusing. What does it mean for p(dtk|w) to be well approximated by  r(w;dtk)?
- 3.3 'In order to reduce the variance of the Monte-Carlo approximation'. What is the MC approximation made here?
- The method never actually uses the assumption that the datapoints are sampled i.i.d.. The algorithm should still work if the datapoints are not examined in a random order, e.g. consider seeing all the images with label '0' and then all the images with label '1' etc.. This would likely degrade the performance significantly but the method should still work.
Overall assessment:
Pros: I like the ideas in the paper and they are presented well.
Cons: The paper is a bit too long. The experiments could more thoroughly investigate the source of the gains.