Regularizing Trajectories to Mitigate Catastrophic Forgetting
Regularization-based continual learning approaches generally prevent catastrophic forgetting by augmenting the training loss with an auxiliary objective. However in most practical optimization scenarios with noisy data and/or gradients, it is possible that stochastic gradient descent can inadvertently change critical parameters. In this paper, we argue for the importance of regularizing optimization trajectories directly. We derive a new co-natural gradient update rule for continual learning whereby the new task gradients are preconditioned with the empirical Fisher information of previously learnt tasks. We show that using the co-natural gradient systematically reduces forgetting in continual learning. Moreover, it helps combat overfitting when learning a new task in a low resource scenario.
The paper puts forward a new regularization based continual learning method that explicitly regularizes the optimization trajectory by constraining in the distribution space. The paper is well written, and preliminary empirical results are promising. [Model] After surveying previous work, I am not sure if the paper is really novel: First of all, adding KL-based constrains to alleviate model forgetting has already been widely explored in prior arts, as the authors also acknowledged in the paper. The proposed regularizer has a close relationship with the EWC, especially with the EWC++. EWC++ encourages the KL-divergence between two distributions learned at successive tasks to be minimized, while the authors proposed to regularize the KL-divergence between two nearby updates, which leads to the well-known natural gradient descent. Natural gradient updates require to calculate the Fisher based on the current curvature, which is computationally expensive in practice. The authors further proposed to use a static Fisher estimated at the previous task for fast approximation. However, the approximation makes the algorithm not a natural gradient descent approach, nor a valid KL-regularized optimization problem. The theoretical implications of the static Fisher approximation are not discussed in the paper. [Experiments] The authors may consider comparing with EWC++, which is a closely related baseline. The experiment results have shown that the co-nature gradient method help from time to time. Understandably, the co-nature gradient-based optimization has the add-on benefit for any continual learning tasks. Still, it would be much better to see how it can help more state-of-the-art methods like LwM, LwF, etc, and especially on a few more challenging datasets.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Regularizing Trajectories to Mitigate Catastrophic Forgetting
Regularization-based continual learning approaches generally prevent catastrophic forgetting by augmenting the training loss with an auxiliary objective. However in most practical optimization scenarios with noisy data and/or gradients, it is possible that stochastic gradient descent can inadvertently change critical parameters. In this paper, we argue for the importance of regularizing optimization trajectories directly. We derive a new co-natural gradient update rule for continual learning whereby the new task gradients are preconditioned with the empirical Fisher information of previously learnt tasks. We show that using the co-natural gradient systematically reduces forgetting in continual learning. Moreover, it helps combat overfitting when learning a new task in a low resource scenario.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The paper puts forward a new regularization based continual learning method that explicitly regularizes the optimization trajectory by constraining in the distribution space. The paper is well written, and preliminary empirical results are promising. [Model] After surveying previous work, I am not sure if the paper is really novel: First of all, adding KL-based constrains to alleviate model forgetting has already been widely explored in prior arts, as the authors also acknowledged in the paper. The proposed regularizer has a close relationship with the EWC, especially with the EWC++. EWC++ encourages the KL-divergence between two distributions learned at successive tasks to be minimized, while the authors proposed to regularize the KL-divergence between two nearby updates, which leads to the well-known natural gradient descent. Natural gradient updates require to calculate the Fisher based on the current curvature, which is computationally expensive in practice. The authors further proposed to use a static Fisher estimated at the previous task for fast approximation. However, the approximation makes the algorithm not a natural gradient descent approach, nor a valid KL-regularized optimization problem. The theoretical implications of the static Fisher approximation are not discussed in the paper. [Experiments] The authors may consider comparing with EWC++, which is a closely related baseline. The experiment results have shown that the co-nature gradient method help from time to time. Understandably, the co-nature gradient-based optimization has the add-on benefit for any continual learning tasks. Still, it would be much better to see how it can help more state-of-the-art methods like LwM, LwF, etc, and especially on a few more challenging datasets.
assistant
The paper puts forward a new regularization based continual learning method that explicitly regularizes the optimization trajectory by constraining in the distribution space. The paper is well written, and preliminary empirical results are promising. [Model] After surveying previous work, I am not sure if the paper is really novel: First of all, adding KL-based constrains to alleviate model forgetting has already been widely explored in prior arts, as the authors also acknowledged in the paper. The proposed regularizer has a close relationship with the EWC, especially with the EWC++. EWC++ encourages the KL-divergence between two distributions learned at successive tasks to be minimized, while the authors proposed to regularize the KL-divergence between two nearby updates, which leads to the well-known natural gradient descent. Natural gradient updates require to calculate the Fisher based on the current curvature, which is computationally expensive in practice. The authors further proposed to use a static Fisher estimated at the previous task for fast approximation. However, the approximation makes the algorithm not a natural gradient descent approach, nor a valid KL-regularized optimization problem. The theoretical implications of the static Fisher approximation are not discussed in the paper. [Experiments] The authors may consider comparing with EWC++, which is a closely related baseline. The experiment results have shown that the co-nature gradient method help from time to time. Understandably, the co-nature gradient-based optimization has the add-on benefit for any continual learning tasks. Still, it would be much better to see how it can help more state-of-the-art methods like LwM, LwF, etc, and especially on a few more challenging datasets.