Approaches to continual learning aim to successfully learn a set of related tasks that arrive in an online manner. Recently, several frameworks have been developed which enable deep learning to be deployed in this learning scenario. A key modelling decision is to what extent the architecture should be shared across tasks. On the one hand, separately modelling each task avoids catastrophic forgetting but it does not support transfer learning and leads to large models. On the other hand, rigidly specifying a shared component and a task-specific part enables task transfer and limits the model size, but it is vulnerable to catastrophic forgetting and restricts the form of task-transfer that can occur. Ideally, the network should adaptively identify which parts of the network to share in a data driven way. Here we introduce such an approach called Continual Learning with Adaptive Weights (CLAW), which is based on probabilistic modelling and variational inference. Experiments show that CLAW achieves state-of-the-art performance on six benchmarks in terms of overall continual learning performance, as measured by classification accuracy, and in terms of addressing catastrophic forgetting.
This paper introduces CLAW, a complex but effective approach to continual learning with strong performance in the sequential task learning setting, as demonstrated on a number of standard benchmarks.
I recommend acceptance because:
- While conceptually similar to VCL, CLAW is convincingly shown to have superior performance across standard benchmarks and measures. The evaluation is thorough across the board, as far as I can tell.
- Forward transfer is shown to be substantially better compared to other methods. Experiments with long sequences of tasks (Omniglot, CIFAR-100) are particularly telling.
- Overall, the balance between not forgetting and still learning new tasks seems particularly favourable for the proposed method. This has been an elusive goal of continual learning research, hence the importance of accepting this work.
Here are some good reasons why an adversarial reviewer would reject this paper:
- The sequential task setting for continual learning has very little to add in practice, and to other branches of machine learning: it has little to say for domains where continual learning problems occur naturally, such as reinforcement learning, GAN training, multi-agent learning; all of these domains need continual learning solutions; while progress on standard benchmarks is important, we may be overfitting to these benchmarks.
- The paper is well written but the method is rather complex and presumably non-trivial to tune. This is actually characteristic of several top competing methods on these benchmarks; getting the last bit of performance seems to require this complexity, but it also makes it that much harder to generalize such methods beyond these benchmarks. For example, exploiting well partitioned datasets into different tasks and known task labels is a good starting point, but once such information is not available all bets are off. Acceptance means encouraging work on these benchmarks; is this really what we should do?
- It's perfectly tractable to store some small amount of old data for these problems in an 'episodic memory', so one could claim that an entire class of relevant baselines is missing, e.g. A-GEM, iCaRL, etc.
Luckily, I am not an adversarial reviewer, but I want to see progress across a more diverse and widely relevant set of continual learning challenges.