Despite recent advances in deep learning, neural networks suffer catastrophic forgetting when tasks are learned sequentially. We propose a conceptually simple and general framework for continual learning, where structure optimization is considered explicitly during learning. We implement this idea by separating the structure and parameter learning. During structure learning, the model optimizes for the best structure for the current task. The model learns when to reuse or modify structure from previous tasks, or create new ones when necessary. The model parameters are then estimated with the optimal structure. Empirically, we found that our approach leads to sensible structures when learning multiple tasks continuously. Additionally, catastrophic forgetting is also largely alleviated from explicit learning of structures. Our method also outperforms all other baselines on the permuted MNIST and split CIFAR datasets in continual learning setting.
The proposed approach aims to mitigate catastrophic forgetting in continual learning (CL) problems by structure learning: determining whether to reuse or adapt existing parameters, or initialise new ones, when faced with a new task. This is framed as an architecture search problem, applying ideas from Differentiable Architecture Search (DARTS). The approach is verified on the Permuted MNIST dataset and evaluated on the Visual Decathlon, showing an improvement.
I think this is an interesting idea with potential, and is worth exploring, and the paper is well-structured and easy to follow.
Unfortunately, I feel the paper fails to consider recent work on CL, both in terms of discussion and benchmarking. The only previous work that is compared is EWC, on permuted MNIST, and the Visual Decathlon performance is only compared to simple baselines (such as adding an adapter or fine tuning) which makes it difficult to gauge the contribution.
There are recent works, some with better results on more difficult problems, such as Variational Continual Learning [1], Progress and Compress [2], or (Variational) Generative Experience Replay [3][4].
Given the approach is based on dynamically adding parameters or modules, Progressive Networks and Dynamically Expandable Networks (both cited) are especially relevant and should be compared (I believe the former may be related to the “adapter” baseline, but this should be made explicit).
I have some questions / discussion points:
- What's the intuition behind implementing the “adapt” operator as additive bias over the previous weights, rather than just copying the previous weights and fine tuning?
- In the general case, if the architecture search is a continuous relaxation (softmax combination of operators), why is the "adapt" operator necessary? Wouldn't this already be a linear combination of new and old parameters? (In the example case of a 1×1 adaptor it makes sense, but this is a special restricted case which adapts with a smaller set of parameters)
- How is the structure regulariser backpropagated into the parameters of each layer? As I understand, it is composed of a constant discrete term z (number of parameters in each option), multiplied by architecture softmaxes alpha; the gradient with respect to each alpha is a constant, and so this has the effect of scaling the gradients of each operator.
- For the "reuse - tuned" case, isn’t the model effectively maintaining a new network for each task?
I also have a number of other comments:
- Reference to figure in page 6 should be figure 4, not 5.
- I think the readability of the paper would benefit from another few proofreads; there are a number of grammatical issues throughout, and several sentence fragments, eg. in the top para of page 2: “..., it has the potential to encourage information sharing. Since now the irrelevant part can be handled…”.
I would encourage the authors to strengthen the experimental comparison by incorporating stronger, external baselines, and improving some of the minor writing issues.
[1] Nguyen, Cuong V., et al. "Variational Continual Learning." ICLR, 2018.
[2] Schwarz, Jonathan, et al. "Progress & Compress: A scalable framework for continual learning." ICML, 2018.
[3] Shin, Hanul, et al. "Continual learning with deep generative replay." NIPS, 2017.
[4] Farquhar, Sebastian, and Yarin Gal. "Towards Robust Evaluations of Continual Learning." arXiv, 2018.