OVERCOMING CATASTROPHIC FORGETTING FOR CONTINUAL LEARNING VIA MODEL ADAPTATION
Learning multiple tasks sequentially is important for the development of AI and lifelong learning systems. However, standard neural network architectures suffer from catastrophic forgetting which makes it difficult for them to learn a sequence of tasks. Several continual learning methods have been proposed to address the problem. In this paper, we propose a very different approach, called Parameter Generation and Model Adaptation (PGMA), to dealing with the problem. The proposed approach learns to build a model, called the solver, with two sets of parameters. The first set is shared by all tasks learned so far and the second set is dynamically generated to adapt the solver to suit each test example in order to classify it. Extensive experiments have been carried out to demonstrate the effectiveness of the proposed approach.
This paper proposes a method for continual learning. The model has three components: a) a data generator to be used at training time to replay past examples, b) a parameter generator that takes the input observation to produce parameters for c) the actual classifier. The authors demonstrate the method on simple datasets with a stream of 2 or 3 tasks.
Strenghts:
- the combination of components is novel
- the method does not rely on task descriptors neither at training nor test time
Weaknesses:
- the paper needs a major rewrite to improve fluency and to better organize and describe the proposed approach
- the empirical validation is weak.
Relevance
Learning in a continual setting is certainly very relevant for this venue.
Novelty
While each component is by itself not very novel (replay methods for continual learning have already been used, networks predicting parameters have also become a fairly common approach in meta-learning literature), the proposed combination is novel in this sub-field.
Clarity
Clarity is very poor and definitely does not meet the acceptance bar for this conference. I believe that the authors would need to make a major revision to address this issue. While ICLR allows authors to revise papers, I think the revision needed to fix this draft goes beyond the acceptable limit, as reviewers would then need to make a whole new revision.
First, fluency is very poor. There are lots of grammatical errors (see first sentence of introduction "neural networks suffers.."),  a plethora of un-necessary acronyms which force the reader to go back and forth to figure out what they refer to (MA, DG, DPG, DPG&S, ...), and several sentences are not well formed (e.g., read first sentence of introduction).
Second, some statements are contradictory; e.g., the authors define "basic unit" as "simple MLP with one hidden layer", but then say it "is an activation function plus a matrix transformation"..
Third, graphics and formulas are too small and not legible.
Fourth, the organization of the paper is poor, it is very wordy yet vague. For instance, the authors should precisely describe how the data generator is trained in sec. 2.3. The authors should provide an algorithm summarizing how the different components interplay both at training and test time. At present, I am making educated guesses about how this system works.
For instance, how are real and generated examples interleaved? how is forgetting prevented in the data generator?   
References to prior work
While there are lots of references in sec. 4, they are not sufficiently well described - see third paragraph of sec. 4 where the authors cite almost 20 papers by simply saying they are "some other approaches". 
Also, I did not find mention to methods predicting parameters in the meta-learning community but also others like:
Denil et al. "Predicting network parameters in deep learning" NIPS 2013
 
Empirical validation
The empirical evaluation does show an advantage of the proposed approach on some simple streams composed by up to three tasks. However, a) the tasks are really simple because of the small number of tasks considered and b) the baselines are weak. For instance, EWC is now a bit out-dated as there are variants that work a bit better, like:
Chaundry et al. "Riemannian Walk ..." ECCV 2018
and there are other methods like "Progress and Compress" Schwartz ICML 2018 the authors could have compared against.
Besides, the authors do not mention anything about memory and time cost both at training and test time, possibly including the time to cross validate all the hyper-parameters of this method.
Overall, I am left with the sense that the proposed approach will be hard to scale to many more tasks and more realistic images (for which we do not quite know how to train efficiently and well generators).
Other comments
I did not find the formalization in eq. 9 very useful. The first and last term in that equation can be very big and there is no sense of how lose this bound is.
Also, it is not clear whether there is a general principle to partition the set of parameters (to determine which ones should be shared).