Published as a conference paper at ICLR 2018 MEMORY-BASED PARAMETER ADAPTATION
Deep neural networks have excelled on a wide range of problems, from vision to language and game playing. Neural networks very gradually incorporate information into weights as they process data, requiring very low learning rates. If the training distribution shifts, the network is slow to adapt, and when it does adapt, it typically performs badly on the training distribution before the shift. Our method, Memory-based Parameter Adaptation, stores examples in memory and then uses a context-based lookup to directly modify the weights of a neural network. Much higher learning rates can be used for this local adaptation, reneging the need for many iterations over similar data before good predictions can be made. As our method is memory-based, it alleviates several shortcomings of neural networks, such as catastrophic forgetting, fast, stable acquisition of new knowledge, learning with an imbalanced class labels, and fast learning during evaluation. We demonstrate this on a range of supervised tasks: large-scale image classification and language modelling.
This article introduces a new method to improve neural network performances on tasks ranging from continual learning (non-stationary target distribution, appearance of new classes, adaptation to new tasks, etc) to better handling of class imbalance, via a hybrid architecture between nearest neighbours and neural net.
After an introduction summarizing their goal, the authors introduce their Model-based parameter adaptation: this hybrid architecture enriches classical deep architectures with a non-parametric “episodic” memory, which is filled at training time with (possibly learned) encodings of training examples and then polled at inference time to refine the neural network parameters with a few steps of gradient in a direction determined by the closest neighbours in memory to the input being processed.  The authors justify this inference-time SGD update with three different interpretations: one linked in Maximum A Posteriori optimization, another to Elastic Weight Regularisation (the current state of the art in continual learning), and one generalising attention mechanisms (although to be honest that later was more elusive to this reviewer). The mandatory literature review on the abundant recent uses of memory in neural networks is then followed by experiments on continual learning tasks involving permuted MNIST tasks, ImageNET incremental inclusion of classes, ImageNet unbalanced, and two language modeling tasks. 
This is an overall very interesting idea, which has the merit of being rather simple in its execution and can be combined with many other methods: it is fully compatible with any optimiser (e.g. ADAM) and can be tacked on top of EWC (which the authors do). The justification is clear, the examples reasonably thorough. It is a very solid paper, which this reviewer believes to be of real interest to the ICLR community.
The following important clarifications from the authors could make it even better:
*  Algorithm 1 in its current form seems to imply an infinite memory, which the experiments make clear is not the case. Therefore: how does the algorithm decide what entries to discard when the memory fills up?
* In most non-trivial settings, the parameter $gamma$ of the encoding is learned, and therefore older entries in the memory lose any ability to be compared to more recent encodings. How do the authors handle this obsolescence of the memory, other than the trivial scheme of relying on KNN to only match recent entries?
* Because gamma needs to be “recent”, this means “theta” is also recent: could the authors give a good intuition on how the two sets of parameters can evolve at different enough timescales to really make the episodic memory relevant? Is it anything else than relying on the fact that the lower levels of a neural net converge before the upper levels?
* Table 1:  could the authors explain why the pre-trained Parametric (and then Mixture) models have the best  AUC in the low-data regime, whereas MbPA was designed very much to be superior in such regimes?
* Paragraph below equation (5), page 3: why not including the regularisation term, whereas the authors just went to great pain to explain it? Rationale? Not including it is also akin to using an improper non-information prior on theta^x independent of theta, which is quite a strong choice to be made “by default”.
* The extra complexity of choosing the learning rate alpha_M and the number of  MpAB steps is worrying this reviewer somewhat. In practice, in Section 4.1the authors explain using grid search to tune the parameters. Is this reviewer correct in understanding that this search is done across all tasks, as opposed to only the first task? And if so, doesn’t this grid search introduce an information leak by bringing information from the whole pre-determined set of task, therefore undermining the very “continuous learning” aim? How do the algorithm performs if the grid search is done only on the first task?
* Figure 3:  the text could clarify that the accuracy is measured across all tasks seen so far. It would be interesting to add a figure (in the Appendix) showing the evolution of the accuracy *per task*, not just the aggregated accuracy. 
* In the related works linking neural networks to encoded episodic memory, the authors might want to include the stream of research on HMAX of Anselmi et al 2014 (https://arxiv.org/pdf/1311.4158.pdf) , Leibo et al 2015 (https://arxiv.org/abs/1512.08457), and Blundell et al 2016 (https://arxiv.org/pdf/1606.04460.pdf ).
Minor typos:
* Figure 4: the title of the key says “New/Old” but then the lines read, in order, “Old” then “New” -- it would be nicer to have them in the same order.
* Section 5: missing period between "ephemeral gradient modifications" and "Further".
* Section 4.2, parenthesis should be "perform well across all 1000 classes", not "all 100 classes".
 
With the above clarifications, this article could become a very remarked contribution.