Artificial neural networks suffer from catastrophic forgetting when they are sequentially trained on multiple tasks. To overcome this problem, we present a novel approach based on task-conditioned hypernetworks, i.e., networks that generate the weights of a target model based on task identity. Continual learning (CL) is less difficult for this class of models thanks to a simple key feature: instead of recalling the input-output relations of all previously seen data, task-conditioned hypernetworks only require rehearsing task-specific weight realizations, which can be maintained in memory using a simple regularizer. Besides achieving state-ofthe-art performance on standard CL benchmarks, additional experiments on long task sequences reveal that task-conditioned hypernetworks display a very large capacity to retain previous memories. Notably, such long memory lifetimes are achieved in a compressive regime, when the number of trainable weights is comparable or smaller than target network size. We provide insight into the structure of low-dimensional task embedding spaces (the input space of the hypernetwork) and show that task-conditioned hypernetworks demonstrate transfer learning. Finally, forward information transfer is further supported by empirical results on a challenging CL benchmark based on the CIFAR-10/100 image datasets.
Paper 1872
Paper proposes a method for CL. The method is based on hypernetworks. These networks are a metamodel, which produce the parameters (from a task-conditioned embedding) which will be used in the main network. Preventing forgetting in the main network is now, replaced by preventing forgetting in the hypernetwork. This is done by imposing a regularization on the hypernetwork outcome, imposing that the generated weights should be similar for previous tasks (similar to Li & Hoiem who impose this on the network outputs). In addition, the paper proposes chunking, which refers to using an additional set of chunk, embeddings which are shared for all tasks, which allow compressing the hypernetwork. Furthermore, they propose an extension that allows for image replay (this is not an easy extension and an impressive contribution on itself, but maybe confusing for the current paper).
CONCLUSION
Overall, I like the idea of the paper and it is well explained. However, I found that the experiments of the paper where not well designed to verify the main contribution (hypernetworks), nor where they compared to the most relevant methods. I am borderline with this paper, and recommend borderline accept (borderline not being an option).
QUESTIONS
1. I think the motivation of why hypernetworks are expected to have less forgetting (than addressing forgetting directly in a network) should be discussed early in the paper. 
2.I do not understand why the training is performed in two steps. First computing a candidate Delta THETA_H and then o ptimizing Eq 2. Why not directly optimizing Eq 2, replacing the second factor with|| f_h(e^t, THETA^*_h)-f_h(e^t, THETA_h) ||. This is how this regularization is normally applied (e.g. Li & Hoiem). If the authors insist in using Eq 2, I would like to see it compared with the proposed version. 
3. The experiments should show that hypernets better address CL then addressing this directly in the network (and preferably provide reasons for this). Comparison with the closest methods like HAT and PackNet should be included. Especially, HAT is interesting since it is also based on an embedding. 
4. Also, more experiments on CIFAR would be welcome. The MNIST variations already provide very high accuracies. For CIFAR-10/100 groups of 20 classes are added in 5 steps ? Scenario CL3 would be interesting for CIFAR as well. 
5. I would like to see more analysis and results for the chunking. (As said before the replay is also a nice addition, but it seems an add-on of the main-text, shrinking the space to analyze the main contributions of the paper in the experiments.)
I guess HNET+ENT for CL1 scenario does not use ENT and is just HNET?