Sequential learning of multiple tasks in artificial neural networks using gradient descent leads to catastrophic forgetting, whereby previously learned knowledge is erased during learning of new, disjoint knowledge. Here, we propose a new approach to sequential learning which leverages the recent discovery of adversarial examples. We use adversarial subspaces from previous tasks to enable learning of new tasks with less interference. We apply our method to sequentially learning to classify digits 0, 1, 2 (task 1), 4, 5, 6, (task 2), and 7, 8, 9 (task 3) in MNIST (disjoint MNIST task). We compare and combine our Adversarial Direction (AD) method with the recently proposed Elastic Weight Consolidation (EWC) method for sequential learning. We train each task for 20 epochs, which yields good initial performance (99.24% correct task 1 performance). After training task 2, and then task 3, both plain gradient descent (PGD) and EWC largely forget task 1 (task 1 accuracy 32.95% for PGD and 41.02% for EWC), while our combined approach (AD+EWC) still achieves 94.53% correct on task 1. We obtain similar results with a much more difficult disjoint CIFAR10 task (70.10% initial task 1 performance, 67.73% after learning tasks 2 and 3 for AD+EWC, while PGD and EWC both fall to chance level). We confirm qualitatively similar results for EMNIST with 5 tasks and under 3 variants of our approach. Our results suggest that AD+EWC can provide better sequential learning performance than either PGD or EWC.
The paper is about a new method for training neural networks in the continual learning setting, where tasks are presented in a sequential manner (and data from the previous task cannot be revisited). The method proposes a new architecture that adds task-parameters parameters to prevent catastrophic forgetting.
To my understanding, the paper proposes a modification to EWC in which the capacity of the network is augmented after a new task is added. Unlike similar methods (like Progressive networks, see bellow), this augmentation is input agnostic. It acts as a correction of the model parameters such that the new task can be easier to train while still maintaining the 'normal parameters' close to the ones of the initial task (as in EWC). I find this idea interesting and certainly worth publishing. In my view, the paper cannot be published in its current state. With the current presentation it is very difficult to understand what is being proposed. Please correct me if I misunderstood the work. 
The writing of the manuscript needs significant improvement. I read it carefully several times and I am still not sure of how exactly the model is trained. I had to read the paper by Elsayed et al 2018, to have an idea of what could have been proposed here. As I mentioned, the paper has novel and interesting ideas, but it would be greatly improved with some important re-writing. Please find bellow some questions.
- In the second to last paragraph of page two, the authors say that: instead of adding a perturbation that would force the network to perform a misclassification, tune it using "the input's own correct class to assist correct classification". If the gradients are computed with respect to the correct class of a given input, why is this called an adversarial perturbation? 
- Elsayed maintain the parameters of the first task fixed and train a fixed input-agnostic correction that can be added to the input such that a second task can be trained (with a re-mapping the outputs). Applying Elsayed et al 2018 to the continual learning setting, the model should only learn correction for task 2 (and 3). How do the authors compute the corrections for task 1? Computing a correction requires having access to the training data.
- The authors use the FGSM method to compute "adversarial perturbations". This method was proposed as a proxy for performing gradient descent to minimize the computational load required for finding adversarial examples. In this application, unlike the case of adversarial perturbations, the memories don't need to be constrained to be smaller than a given epsilon. What is the motivation of using this method? How do you explain the difference in the results.
- Having mentioned this, both W_task and M_task are updated by minimizing the same loss function (ignoring the difference of using FGSD or not). In that case, why is it needed to have a factorized form W_task * M_task instead of a single bias?
- Throughout the paper the authors say that the long term memory lies on the "intersection of adversarial subspaces". It is not clear at all why this should be the case. The authors do not explain adversarial subspaces corresponding to which model.
- The authors should cite the Progressive networks as this is a very related work. Unlike progressive networks, this work proposes and augmentation that is input agnostic which is interesting. https://arxiv.org/abs/1606.04671
- With EWC, once the model is trained, one does not need to know the task being evaluated at test time. This is not the case in the proposed model. This should be clarified. Also, when having many tasks mapping to the same input, the fair way of comparing to EWC would to have a different head per task. This baselines should be included.
- What are the task specific functions g_taskA and g_taskB?
- Adding an explicit algorithm, the exact loss functions used should help clarifying the proposed method.
- The paper would be stronger if more complex tasks would be added.