Lifelong learning aims at avoiding the catastrophic forgetting problem of traditional supervised learning models. Episodic memory based lifelong learning methods such as A-GEM (Chaudhry et al., 2018b) are shown to achieve the stateof-the-art results across the benchmarks. In A-GEM, a small episodic memory is utilized to store a random subset of the examples from previous tasks. While the model is trained on a new task, a reference gradient is computed on the episodic memory to guide the direction of the current update. While A-GEM has strong continual learning ability, it is not clear that if it can retain the performance in the presence of adversarial attacks. In this paper, we examine the robustness of A-GEM against adversarial attacks to the examples in the episodic memory. We evaluate the effectiveness of traditional attack methods such as FGSM and PGD. The results show that A-GEM still possesses strong continual learning ability in the presence of adversarial examples in the memory and simple defense techniques such as label smoothing can further alleviate the adversarial effects. We presume that traditional attack methods are specially designed for standard supervised learning models rather than lifelong learning models. we therefore propose a principled way for attacking A-GEM called gradient reversion (GREV) which is shown to be more effective. Our results indicate that future lifelong learning research should bear adversarial attacks in mind to develop more robust lifelong learning algorithms.
This paper does a good job of raising awareness of adversarial attacks in lifelong learning research with deep neural networks. This is the first time I have considered this problem, but not sure whether any prior work exists in the specific subfield.
At the conceptual level, many issues can arise when a lifelong learner is attacked, since systematic negative bias could be introduced in the training process and may be very difficult to remove, given the tendency to 'remember everything' which dominates current approaches.
The paper isolates one lifelong learning approach (A-GEM) which is characteristic of one (of many) different approaches to lifelong learning, and investigates its robustness to standard adversarial attacks and a novel attack developed within this paper, which is stronger, but specific to episodic memory approaches.
I cannot recommend acceptance at this point for the following reasons:
1) I am not sure what I can generalize away from this paper to the immediate subfield and beyond. The paper claims that the investigated method is SOTA, but it's not clear this is the case, even in restricted class of similar episodic memory based models, see [1] for an independent evaluation of many such approaches. Is there any reasons why conclusions about this particular method are indeed representative of its class?
2) While the paper does not explicitly make this claim, the title suggests that 'gradient reversion' attacks apply to lifelong learning models in general. Why is this class of approaches particularly informative such that conclusions may hold in general? Are other methods in this class more susceptible to these attacks and can the proposed attack be applied to the whole class, or even other types of approaches? This should be clarified!
References
[1] Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Gregory Slabaugh, Tinne Tuytelaars,  Continual learning: A comparative study on how to defy forgetting in classification tasks, https://arxiv.org/abs/1909.08383