Continual learning aims to learn new tasks without forgetting previously learned ones. This is especially challenging when one cannot access data from previous tasks and when the model has a fixed capacity. Current regularization-based continual learning algorithms need an external representation and extra computation to measure the parameters' importance. In contrast, we propose Uncertaintyguided Continual Bayesian Neural Networks (UCB), where the learning rate adapts according to the uncertainty defined in the probability distribution of the weights in networks. Uncertainty is a natural way to identify what to remember and what to change as we continually learn, and thus mitigate catastrophic forgetting. We also show a variant of our model, which uses uncertainty for weight pruning and retains task performance after pruning by saving binary masks per tasks. We evaluate our UCB approach extensively on diverse object classification datasets with short and long sequences of tasks and report superior or on-par performance compared to existing approaches. Additionally, we show that our model does not necessarily need task information at test time, i.e. it does not presume knowledge of which task a sample belongs to.
** post rebuttal start **
After reading reviews and authors' response, I decided not to change my score.
I am happy with the author's response addressing my concerns (mainly about the fairness on the size of the model), so I recommend its acceptance. I believe it is a good addition to the community of continual learning.
** post rebuttal end **
- Summary:
This paper proposes to use a way to improve continual learning performance by taking "Bayes-by-backprop" method. They claim that the uncertainty can naturally be measured by estimating (log of) the standard deviation, and it is indeed useful to judge the importance of each learnable parameter. Experimental results on several benchmarks show that their method outperforms few state-of-the-art methods.
- Decision and supporting arguments:
Weak accept.
1. The proposed method is simple but effective. However, It is still questionable whether \sigma is the best measure of the weight importance. An ablation study with different choices of the importance measure (maybe \mu can also be incorporated as well as \sigma?) would be good to see.
2. Survey and comparison with memory-based methods are limited. Though memory-based methods require some memory to keep the experience, the proposed method also requires additional memory for \sigma; it essentially doubles the model capacity, assuming that \sigma is solely for measuring the weight importance. In particular, when it comes to large-scale models, memory for storing some important experiences would be small compared to the memory to store the model.
Here are some papers about recently proposed memory-based methods, which are not cited:
Castro et al. End-to-End Incremental Learning. In ECCV, 2018.
Wu et al. Large Scale Incremental Learning. In CVPR, 2019.
Lee et al. Overcoming Catastrophic Forgetting with Unlabeled Data in the Wild. In ICCV, 2019.
3. Comparison should include the model capacity as in Table 1(b). Again, compared to the conventional non-Bayesian model, half of the model capacity is used for computing \sigma (uncertainty), I wonder it causes a performance drop when the model capacity is the same over all compared methods. If they used the same model architecture and just doubled the number of learnable parameters for \sigma, then it is obviously unfair.
- Comments:
1. Pruning is not beneficial in terms of the performance. I hope to see some quantitative benefits obtained by introducing pruning. In Table 1(b), why doesn't pruning reduce the number of parameters?