EFFICIENT LIFELONG LEARNING WITH A-GEM
In lifelong learning, the learner is presented with a sequence of tasks, incrementally building a data-driven prior which may be leveraged to speed up learning of a new task. In this work, we investigate the efficiency of current lifelong approaches, in terms of sample complexity, computational and memory cost. Towards this end, we first introduce a new and a more realistic evaluation protocol, whereby learners observe each example only once and hyper-parameter selection is done on a small and disjoint set of tasks, which is not used for the actual learning experience and evaluation. Second, we introduce a new metric measuring how quickly a learner acquires a new skill. Third, we propose an improved version of GEM (Lopez-Paz & Ranzato, 2017), dubbed Averaged GEM (A-GEM), which enjoys the same or even better performance as GEM, while being almost as computationally and memory efficient as EWC ) and other regularizationbased methods. Finally, we show that all algorithms including A-GEM can learn even more quickly if they are provided with task descriptors specifying the classification tasks under consideration. Our experiments on several standard lifelong learning benchmarks demonstrate that A-GEM has the best trade-off between accuracy and efficiency.
Summary of the paper:
This paper focuses on the problem of lifelong learning for multi-task
neural networks. The goal is to learn in a computationally and memory
efficient manner new tasks as they are encountered while at the same
time remembering how to solve previously seen tasks with a focus on
having only one training pass through all the training data. The paper
builds on the GEM method introduced in the paper "Gradient episodic
memory for continuum learning", NIPS 2017.
The main novelty over the original GEM paper is that A-GEM simplifies
the constraints on what constitutes a feasible update step during its
SGD training so that GEM's QP problem is replaced by a couple of
inner-products (and thus makes A-GEM much more computationally
efficient). This simplification also means that only one gradient
vector (the average gradient computed from the individual gradients of
the task loss of the previously seen tasks) has to be stored at each
update as opposed to GEM where each task specific gradient vector has
to be stored. Thus the memory requirements of A-GEM is much less than
GEM and is independent of the number of already learnt tasks.
The paper then presents experimental evidence that A-GEM does run much
faster and uses less memory and results in performance similar to the
original GEM strategy. The latter point is important as the simplified
A-GEM algorithm - which adjusts the network's parameter to improve
performance on the current task while ensuring the average performance
on the previously seen tasks should not decrease - does not guarantee
as stringently as GEM that the network does not forget how to perform
all the previous tasks.
The paper also introduces an extra performance metric is introduced
  called the "Learning Curve Area" which measures how quickly a new
  task is learnt when it is presented with new material.
Pros and Cons of the paper:
+/- The paper presents a simple intuitive extension to the original GEM
paper that is much more computationally efficient and is thus more
suited and feasible for real lifelong learning applications. And it
shows that performance exceeds other methods that have similar
computational demands. The paper can be viewed as somewhat incremental
but the increment is probably crucial for any real-world practical
application.
+ The validity of the approach is demonstrated experimentally on
  standard datasets in the field.
- Some of the presentation of the material is somewhat vague, in
  particular section 5. In this section a joint embedding model is
  described that helps facilitate zero-shot learning. However, not
  enough detail is given to fully understand or appreciate this
  contribution, see below for details.
Rationale for my evaluation:
The method is somewhat incremental, however, this increment could be
quite practically important. The presentation is lacking in some regard and would benefit
 from some re-working i.e. section 5. 
 
Unclear in the paper:
Section 5 describing the "Joint Embedding Model Using Compositional Task descriptors" is very sparse on detail.  Here are some of the details that I feel are missing:
- In the experiments how is the matrix description (via attributes) of the different tasks $t^k$ learnt/discovered?
- The size of this attribute matrix is able to vary from one task to the next. How does the function $\psi_{\omega}$ deal with this problem?
- What functions are used in the experiments to represent $\psi_{\omega}$?
- In the second last line of paragraph 2 should $C$ be $C_k$? If it should be $C$ how is $C$ chosen?
- In equation (12) should the $c$th column of $\psi_{\omega}$ be extracted as opposed to the $k$th column?
Representative labelled samples from each task are stored in memory
and these are used when learning for a new task. The system
has a fixed memory so when a new task is added then the number of
images stored for each task has to be reduced. Then uniform sampling is
used to randomly decide which images to keep. Could this selection
process be improved upon and would any such improvement have any large
impact on performance?
Typos and minor errors spotted:
In the third paragraph of section 2 it is stated $T^{CV} \ll T$ in the
experiments performed this is not case. I don't think 3 is much less
than 10 or 20.
In figures 4 and 5 it is not entirely clear which curves correspond to
A-GEM and A-GEM-JE from the legend. In the legend the dashed line with
the triangle looks the same the non-dashed line with the triangle. I
presuming A-GEM is the non-dashed line, but only because that makes
things consistent with the previous figures.