Current deep neural networks can achieve remarkable performance on a single task. However, when the deep neural network is continually trained on a sequence of tasks, it seems to gradually forget the previous learned knowledge. This phenomenon is referred to as catastrophic forgetting and motivates the field called lifelong learning. The central question in lifelong learning is how to enable deep neural networks to maintain performance on old tasks while learning a new task. In this paper, we introduce a novel and effective lifelong learning algorithm, called MixEd stochastic GrAdient (MEGA), which allows deep neural networks to acquire the ability of retaining performance on old tasks while learning new tasks. MEGA modulates the balance between old tasks and the new task by integrating the current gradient with the gradient computed on a small reference episodic memory. Extensive experimental results show that the proposed MEGA algorithm significantly advances the state-of-the-art on all four commonly used lifelong learning benchmarks, reducing the error by up to 18%.
========================== Summary of paper ==========================
The paper proposes a new loss balancing approach for lifelong learning. The proposed method dynamically balances the old task loss (from an episodic memory) and the new task loss based on their magnitude. The paper outperforms state-of-the-art method (that don't use extra attribute info), and is straightforward to understand.
========================== Decision ==========================
I vote for rejecting the paper. The paper adds a very simple dynamic loss balancing to a joint loss, which has limited novelty, yet does not discuss its relationship with loss balancing in multitask learning. Although the method outperforms state-of-the-art, a very simple baseline of their method also outperforms state-of-the-art, making the contribution ineffective. The writing of the paper may also benefit from further edits.
========================== Pros/Cons ==========================
+ The paper outperforms state-of-the-art.
+ The method is simple to explain and straightforward to implement.
+ Proposes a weighted variant of forgetting measure (although the purpose is not justified -- need to discuss what use case would make one prefer this weighted variant better than the original).
- The paper is not well-placed in the literature. Not until the bottom of page 5 can readers see very closely related methods, and despite the similarity, little is discussed about the difference. To improve, relationships with existing multitask learning weight balancing methods (e.g. https://arxiv.org/abs/1810.04650, https://arxiv.org/abs/1705.07115, https://arxiv.org/abs/1711.02257) should also be discussed, and maybe compared to.
- Motivation to use the proposed loss balancing rather than that of very similar methods (e.g. GEM/A-GEM) is lacking ("These approaches cannot capture the dynamics in the lifelong learning process", but GEM's balancing is also dynamic)
- Solving for beta using optimization (eq. 10), while (eq. 9) should have an explicit, close-form solution.
- Writing sometimes is confusing, uses absolute language, or using claims unsupported by evidence.
    - (1) page 3 "In this case, the weights are only optimized for the current task while ignoring previous tasks which leads to catastrophic forgetting." ignores the existence of regularization-based methods such as EWC
    - (2) bottom of page 3 "alpha1 and alpha2 should be adjusted adaptively" does not have experiment results supporting it (especially considering GEM is also adaptive)
    - (3) "l_ref(w; ζ) = 0 implies that there is almost no catastrophic forgetting" claim is problematic. Overfitting to the episodic memory is a common problem.
    - (3) page 3 xi and zeta not clearly defined. 
    - (4) brings up NP-hardness while it is seldom of interest in this field.
    - (5) missing "∇" on the denominator in (eq. 7).
- Experiment:
    - The only ablation (the direct approach) is statistically indistinguishable from the proposed method. This also outperforms state-of-the-art, while it should not be. One can only assume that the experiment is problematic.
    - Completely uses hyperparameter from an unpublished paper.
    - Claims state-of-the-art, yet omits a state-of-the-art variant in cited A-GEM paper (with joint-embedding). This should be discussed even if the comparison is unfair.
========================== Improving the paper ==========================
- Rewrite the introduction so it is clear the paper is an improvement over A-GEM with a better loss balancing, and focus the motivation and side experiments on why this is important for lifelong learning.
- Clean up writing.
- Explain why the direct approach, while being a basic loss balancing method, outperform state-of-the-art GEM greatly. Perhaps a set of ablation studies can help.
- Replace optimizing for beta as solving for beta.
- Comparison with other loss balancing papers.
- Clarify state-of-the-art comparison.
========================== After rebuttal ==========================
I appreciate the authors addressing my concerns about the placement in the literature; it looks clearer now. However, the rebuttal did not address well the question why the simple MEGA-D variant outperforms prior state-of-the-art besides promising to publish code. The new ablation study is especially difficult to understand, since the first set of results are MEGA without using both memory and new data, but it seems to me that MEGA *needs* the memory data to work, so I have no idea what is being ablated here, and I also do not understand how this answers the question why MEGA-D performs so well. I will not be changing my rating.
Other issues:
(1) Citing A-GEM: please update the bib file so it shows the ICLR version instead of arXiv.
(2) Solving eq. 9: Here you go: https://imgur.com/a/b4QCVlv
I literally had this problem on one of my high school exams. Feel free to use this in any future version.