Revisiting Gradient Episodic Memory for Continual Learning
Gradient Episodic Memory (GEM) is an effective model for continual learning, where each gradient update for the current task is formulated as a quadratic program problem with inequality constraints that alleviate catastrophic forgetting of previous tasks. However, practical use of GEM is impeded by several limitations: (1) the data examples stored in the episodic memory may not be representative of past tasks; (2) the inequality constraints appear to be rather restrictive for competing or conflicting tasks; (3) the inequality constraints can only avoid catastrophic forgetting but can not assure positive backward transfer. To address these issues, in this paper we aim at improving the original GEM model via three handy techniques without extra computational cost. Experiments on MNIST Permutations and incremental CIFAR100 datasets demonstrate that our techniques enhance the performance of GEM remarkably. On CIFAR100 the average accuracy is improved from 66.48% to 68.76%, along with the backward (knowledge) transfer growing from 1.38% to 4.03%.
This paper proposes an extension to gradient episodic memory (GEM) to improve its performance and backwards transfer. Specifically, the proposed method selects *support examples* to represent each task (versus the last M examples for GEM); introduces slack variables to ensure the constraints imposed by GEM are not too restrictive; and uses cosine similarity between sample gradients to encourage backwards transfer. The ideas proposed are interesting, and the paper is easy to follow. However, I have a number of concerns which I think, unfortunately, preclude publication at this point. Primarily, while I believe the ideas are intuitive and novel, the experimental evaluation does not appear to support the claim that the proposed method significantly improves GEM (only a minor improvement is shown at best). This is further compounded by the fact that variances over multiple seeds/runs are not reported, which makes it difficult to gauge any statistically significant performance improvement. Some claims in the paper also need to be tempered: - The abstract suggests performance improves *remarkably*, but experiments do not support this. - The last paragraph of Section 4.2 declares that adding a offset to the Lagrangian, as suggested by the GEM authors, *is an ad-hoc practice and lacks rationality: it is not faithfully follow their mathematical formulation*. This may come across as confrontational, and I don*t feel the point is valid in this case, given that the soft gradient constraints introduce a similar trade-off parameter. The language is also imprecise and conversational at times; I would suggest another read-through and careful rewording for clarity. For example, in 4.2.2, *this new constraint imposes cosine similarity actually...*. Lastly, I*m not sure about this, but does the link to the GitHub repository break anonymity? Given the language difference, I*m not sure what the url refers to; but wonder if it could be a name. The ideas have potential, and I suggest the authors explore avenues to further improve the efficacy of the method and expand the experimentation, and improve some of the writing in terms of language and claims made. (A note on the score: I don*t think this is a 1 under the old ICLR system, but unfortunately have to recommend it be rejected in its current state)
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Revisiting Gradient Episodic Memory for Continual Learning
Gradient Episodic Memory (GEM) is an effective model for continual learning, where each gradient update for the current task is formulated as a quadratic program problem with inequality constraints that alleviate catastrophic forgetting of previous tasks. However, practical use of GEM is impeded by several limitations: (1) the data examples stored in the episodic memory may not be representative of past tasks; (2) the inequality constraints appear to be rather restrictive for competing or conflicting tasks; (3) the inequality constraints can only avoid catastrophic forgetting but can not assure positive backward transfer. To address these issues, in this paper we aim at improving the original GEM model via three handy techniques without extra computational cost. Experiments on MNIST Permutations and incremental CIFAR100 datasets demonstrate that our techniques enhance the performance of GEM remarkably. On CIFAR100 the average accuracy is improved from 66.48% to 68.76%, along with the backward (knowledge) transfer growing from 1.38% to 4.03%.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper proposes an extension to gradient episodic memory (GEM) to improve its performance and backwards transfer. Specifically, the proposed method selects *support examples* to represent each task (versus the last M examples for GEM); introduces slack variables to ensure the constraints imposed by GEM are not too restrictive; and uses cosine similarity between sample gradients to encourage backwards transfer. The ideas proposed are interesting, and the paper is easy to follow. However, I have a number of concerns which I think, unfortunately, preclude publication at this point. Primarily, while I believe the ideas are intuitive and novel, the experimental evaluation does not appear to support the claim that the proposed method significantly improves GEM (only a minor improvement is shown at best). This is further compounded by the fact that variances over multiple seeds/runs are not reported, which makes it difficult to gauge any statistically significant performance improvement. Some claims in the paper also need to be tempered: - The abstract suggests performance improves *remarkably*, but experiments do not support this. - The last paragraph of Section 4.2 declares that adding a offset to the Lagrangian, as suggested by the GEM authors, *is an ad-hoc practice and lacks rationality: it is not faithfully follow their mathematical formulation*. This may come across as confrontational, and I don*t feel the point is valid in this case, given that the soft gradient constraints introduce a similar trade-off parameter. The language is also imprecise and conversational at times; I would suggest another read-through and careful rewording for clarity. For example, in 4.2.2, *this new constraint imposes cosine similarity actually...*. Lastly, I*m not sure about this, but does the link to the GitHub repository break anonymity? Given the language difference, I*m not sure what the url refers to; but wonder if it could be a name. The ideas have potential, and I suggest the authors explore avenues to further improve the efficacy of the method and expand the experimentation, and improve some of the writing in terms of language and claims made. (A note on the score: I don*t think this is a 1 under the old ICLR system, but unfortunately have to recommend it be rejected in its current state)
assistant
This paper proposes an extension to gradient episodic memory (GEM) to improve its performance and backwards transfer. Specifically, the proposed method selects *support examples* to represent each task (versus the last M examples for GEM); introduces slack variables to ensure the constraints imposed by GEM are not too restrictive; and uses cosine similarity between sample gradients to encourage backwards transfer. The ideas proposed are interesting, and the paper is easy to follow. However, I have a number of concerns which I think, unfortunately, preclude publication at this point. Primarily, while I believe the ideas are intuitive and novel, the experimental evaluation does not appear to support the claim that the proposed method significantly improves GEM (only a minor improvement is shown at best). This is further compounded by the fact that variances over multiple seeds/runs are not reported, which makes it difficult to gauge any statistically significant performance improvement. Some claims in the paper also need to be tempered: - The abstract suggests performance improves *remarkably*, but experiments do not support this. - The last paragraph of Section 4.2 declares that adding a offset to the Lagrangian, as suggested by the GEM authors, *is an ad-hoc practice and lacks rationality: it is not faithfully follow their mathematical formulation*. This may come across as confrontational, and I don*t feel the point is valid in this case, given that the soft gradient constraints introduce a similar trade-off parameter. The language is also imprecise and conversational at times; I would suggest another read-through and careful rewording for clarity. For example, in 4.2.2, *this new constraint imposes cosine similarity actually...*. Lastly, I*m not sure about this, but does the link to the GitHub repository break anonymity? Given the language difference, I*m not sure what the url refers to; but wonder if it could be a name. The ideas have potential, and I suggest the authors explore avenues to further improve the efficacy of the method and expand the experimentation, and improve some of the writing in terms of language and claims made. (A note on the score: I don*t think this is a 1 under the old ICLR system, but unfortunately have to recommend it be rejected in its current state)