Learning with Long-term Remembering: Following the Lead of Mixed Stochastic Gradient
Current deep neural networks can achieve remarkable performance on a single task. However, when the deep neural network is continually trained on a sequence of tasks, it seems to gradually forget the previous learned knowledge. This phenomenon is referred to as catastrophic forgetting and motivates the field called lifelong learning. The central question in lifelong learning is how to enable deep neural networks to maintain performance on old tasks while learning a new task. In this paper, we introduce a novel and effective lifelong learning algorithm, called MixEd stochastic GrAdient (MEGA), which allows deep neural networks to acquire the ability of retaining performance on old tasks while learning new tasks. MEGA modulates the balance between old tasks and the new task by integrating the current gradient with the gradient computed on a small reference episodic memory. Extensive experimental results show that the proposed MEGA algorithm significantly advances the state-of-the-art on all four commonly used life-long learning benchmarks, reducing the error by up to 18%.
I am very torn about this paper as the experiments are thorough and the results are quite significant. The results are on four popular benchmarks and they also test on Many Permutations, which is an important finding as well. While the angle based approach presented here is interesting and intuitively appealing, I am not really sure whether it is a needed generalization beyond past approaches. I don*t really have a strong intuition for where the gains are coming from. As such, I think the paper would be much stronger if it could intuitively or theoretically get at why it improves on past approaches. Actually, one of my biggest concerns is how good the performance of the direct approach is. The direct approach seems like a straightforward extension of experience replay with a special learning rate for current examples vs. buffer examples that is adaptive to the relative loss. While it is potentially interesting that this approach works so well, it is not positioned as the selling point of the paper as currently written. Additionally, this is kind of an orthogonal contribution to past papers leveraging experience replay, which could potentially be modified and improved based on this approach. I wonder if the authors tried modifying any of the baseline approaches in this way as well. A small note on the experiments: the authors closely follow settings from past work and at times it can be unclear which results were implemented in this paper and which in past papers. I don*t think that you have addressed what hyperparameters you chose for MER. It also seems to perform worse than past reported results on CIFAR and is not reported despite the best past performance on MNIST Permutations.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Learning with Long-term Remembering: Following the Lead of Mixed Stochastic Gradient
Current deep neural networks can achieve remarkable performance on a single task. However, when the deep neural network is continually trained on a sequence of tasks, it seems to gradually forget the previous learned knowledge. This phenomenon is referred to as catastrophic forgetting and motivates the field called lifelong learning. The central question in lifelong learning is how to enable deep neural networks to maintain performance on old tasks while learning a new task. In this paper, we introduce a novel and effective lifelong learning algorithm, called MixEd stochastic GrAdient (MEGA), which allows deep neural networks to acquire the ability of retaining performance on old tasks while learning new tasks. MEGA modulates the balance between old tasks and the new task by integrating the current gradient with the gradient computed on a small reference episodic memory. Extensive experimental results show that the proposed MEGA algorithm significantly advances the state-of-the-art on all four commonly used life-long learning benchmarks, reducing the error by up to 18%.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



I am very torn about this paper as the experiments are thorough and the results are quite significant. The results are on four popular benchmarks and they also test on Many Permutations, which is an important finding as well. While the angle based approach presented here is interesting and intuitively appealing, I am not really sure whether it is a needed generalization beyond past approaches. I don*t really have a strong intuition for where the gains are coming from. As such, I think the paper would be much stronger if it could intuitively or theoretically get at why it improves on past approaches. Actually, one of my biggest concerns is how good the performance of the direct approach is. The direct approach seems like a straightforward extension of experience replay with a special learning rate for current examples vs. buffer examples that is adaptive to the relative loss. While it is potentially interesting that this approach works so well, it is not positioned as the selling point of the paper as currently written. Additionally, this is kind of an orthogonal contribution to past papers leveraging experience replay, which could potentially be modified and improved based on this approach. I wonder if the authors tried modifying any of the baseline approaches in this way as well. A small note on the experiments: the authors closely follow settings from past work and at times it can be unclear which results were implemented in this paper and which in past papers. I don*t think that you have addressed what hyperparameters you chose for MER. It also seems to perform worse than past reported results on CIFAR and is not reported despite the best past performance on MNIST Permutations.
assistant
I am very torn about this paper as the experiments are thorough and the results are quite significant. The results are on four popular benchmarks and they also test on Many Permutations, which is an important finding as well. While the angle based approach presented here is interesting and intuitively appealing, I am not really sure whether it is a needed generalization beyond past approaches. I don*t really have a strong intuition for where the gains are coming from. As such, I think the paper would be much stronger if it could intuitively or theoretically get at why it improves on past approaches. Actually, one of my biggest concerns is how good the performance of the direct approach is. The direct approach seems like a straightforward extension of experience replay with a special learning rate for current examples vs. buffer examples that is adaptive to the relative loss. While it is potentially interesting that this approach works so well, it is not positioned as the selling point of the paper as currently written. Additionally, this is kind of an orthogonal contribution to past papers leveraging experience replay, which could potentially be modified and improved based on this approach. I wonder if the authors tried modifying any of the baseline approaches in this way as well. A small note on the experiments: the authors closely follow settings from past work and at times it can be unclear which results were implemented in this paper and which in past papers. I don*t think that you have addressed what hyperparameters you chose for MER. It also seems to perform worse than past reported results on CIFAR and is not reported despite the best past performance on MNIST Permutations.