Humans excel in continuously learning with small data without forgetting how to solve old problems. However, neural networks require large datasets to compute latent representations across different tasks while minimizing a loss function. For example, a natural language understanding (NLU) system will often deal with emerging entities during its deployment as interactions with users in realistic scenarios will generate new and infrequent names, events, and locations. Here, we address this scenario by introducing a RL trainable controller that disentangles the representation learning of a neural encoder from its memory management role.
Our proposed solution is straightforward and simple: we train a controller to
The authors build on work regarding few-shot learning with memory augmented networks, specifically [Kaiser, et al., ICLR17] where the goal is to learn a memory address mapping such that generalization is achieved by finding the nearest neighbor memory address when predicting the label. For correct predictions, the memory key is updated to include the associated (predicted) address while new memory locations are written for mistakes. Whereas [Kaiser, et al., ICLR17] follows a LRU-like procedure for replacing memory, the current work proposes performing policy-gradient RL where the action space is the memory locations and the reward is reduction of entropy over the memory address assignment distribution over the memory locations. This approach is empirically studied for an RNN approach to NER, specifically considering few-shot learning for NER in the Stanford Task-Oriented Dialogue (STOD) dataset — showing non-negligible improvements over Memory Augmented Networks [Santoro, et al., ICML16] and Matching Networks [Vinyals, et al., NeurIPS16].
From a high-level perspective, memory networks have played an important role in building dialogue managers (and in several applications) and their proposed expansion over [Kaiser, et al., ICLR17] in the context of few-shot, sparse memory architectures seems sensible. Additionally, the empirical results o NER seem promising. However, in my opinion, this paper is making contributions along two dimensions, but neither of them convincingly as detailed below. 
First, one potential point of emphasis is the ‘learning to control’ mechanism through PG RL being better than [Kaiser, et al., ICLR17] using with a LRU-like policy. To do this, it would make sense to both compare to the datasets in the previous work (Omniglot, WMT — even if it does require additional architectures that are pretty straightforward expansions) and to include this architecture in the submitted paper as there is a publicly available dataset (https://github.com/himani-arora/learning_to_remember_rare_events). While the synthetic dataset probably isn’t necessary, one could easily think of synthetic datasets that would clearly contrast with the existing method and conduct ablation/configuration studies regarding different number of neighbors, size of memory, exact vs. approximate nearest neighbor for lookup, etc. 
A second point of interest is the NER task. From this perspective, there is some existing work (e.g., [Fritzler, Logacheva & Kretov, SAC19; Hou, et al., https://arxiv.org/pdf/1906.08711.pdf (Not required as not accepted yet); Hofer, et al., https://arxiv.org/pdf/1811.05468.pdf (Not clear if accepted)]. While these are largely preliminary works, they do provide datasets (and there are many others) for NER that would potentially make a convincing case for this being a state-of-the-art approach for few-shot learning in NER. However, I am not sure that being the state-of-the-art for few-shot NER would be a sufficient contribution for ICLR, even if done convincingly. If there was something in the algorithm specific to sequence prediction, then I think few-shot sequence prediction would be a sufficient contribution, but given the current architecture, I would think that the bar is a methodological contribution to adding sparse memory units (via RL in this case) is the acceptable level of contribution.
Finally, my belief that the contribution is too narrow and not sufficiently developed is supported by the writing seeming rushed. The paper makes much more sense after reading [Kaiser, et al., ICLR17] as Sections 2, 3 of the submission are missing the aspects of a clear constrast to [Kaiser, et al., ICLR17], a working example to provide readers a more intuitive understanding, a clear example showing dimensionality and notation (beyond what is in Figure 1). There are also many typos (e.g., “mammals exel”, “trainale controller”, “Rathern then”, amongst others) and general need for proofreading. However, this is not the determining factor in the paper. Basically, I think the contribution is insufficient in terms of scope and convincingness of the contribution. Thus, I recommend rejecting in its current form — even if the underlying idea is promising and worth developing further.