Gradient Episodic Memory (GEM) is an effective model for continual learning, where each gradient update for the current task is formulated as a quadratic program problem with inequality constraints that alleviate catastrophic forgetting of previous tasks. However, practical use of GEM is impeded by several limitations: (1) the data examples stored in the episodic memory may not be representative of past tasks; (2) the inequality constraints appear to be rather restrictive for competing or conflicting tasks; (3) the inequality constraints can only avoid catastrophic forgetting but can not assure positive backward transfer. To address these issues, in this paper we aim at improving the original GEM model via three handy techniques without extra computational cost. Experiments on MNIST Permutations and incremental CIFAR100 datasets demonstrate that our techniques enhance the performance of GEM remarkably. On CIFAR100 the average accuracy is improved from 66.48% to 68.76%, along with the backward (knowledge) transfer growing from 1.38% to 4.03%.
This paper presents three improvements to the previous GEM algorithm: choosing support examples better (the GEM paper used a random set), incorporating soft gradient constraints, and specifying the magnitude of the dot product in the gradient optimisation problem (in order to increase positive backward transfer). They then compare the original GEM algorithm (and other baselines) with their algorithm on two datasets, consider the case where there is less memory available, and provide ablation studies for their three improvements.
I recommend to reject this paper. I am struggling to see the improvements in the results, as each algorithm is only run once on each dataset, and all the numbers seem very close together (GEM vs new algorithm). There are also a significant number of hyperparameters added by this paper: how are these tuned? I also do not understand the reasoning behind the third idea ('positive backward transfer'). I do, however, like the other two improvements and see the reasoning behind them, however I still have some misgivings. I will now elaborate on these points.
Firstly, there are no standard deviations provided for the experiments. I believe this is especially important to do for the original GEM algorithm and the proposed algorithm because the two provide extremely similar metric values. Comparing values from the original GEM paper and this paper's run of GEM indicates to me that there is a significant chance that any improvements could be within error bars. For example, on CIFAR100, the authors' run of GEM gives 66.48%, the original GEM paper reports 67.83%, and the authors' method is 68.76%.
Secondly, this paper introduces a significant number of hyperparameters over the original GEM algorithm. How are these tuned? Is there a validation set? Does having to tune these hyperparameters slow down computation significantly (how much exactly)? It would be nice to see how much more computationally expensive the new method is compared with GEM: the authors claim "little computational burdens".
I like the soft gradient constraint idea introduced in this paper. It is more principled than the hack that the original GEM paper used. The method of choosing the support set also makes sense to me, however there are many hyperparameters in this idea. Additionally, the results (Table 3) for different memory sizes are confusing. It seems like smaller memory sizes lead to less improvement over random memory. Surely cleverly choosing memory should be more beneficial when constrained to smaller memory? This indicates to me that improvements need to still be made to the idea. I also do not follow the explanation given in Section 4.2.1 for the positive backward transfer improvement. Why should the magnitude of the inner products be specified? What do you mean by "cosine similarity in magnitude"?
As a final comment, I will say that there are many typos in this paper.