We present a novel approach for training neural machines which incorporates additional supervision on the machine's interpretable components (e.g., neural memory). To cleanly capture the kind of neural machines to which our method applies, we introduce the concept of a differential neural computational machine (∂NCM) and show that several existing architectures (e.g., NTMs, NRAMs) can be instantiated as a ∂NCM and can thus benefit from any amount of additional supervision over their interpretable components. Based on our method, we performed a detailed experimental evaluation with NTM and NRAM machines, showing the approach leads to significantly better convergence and generalization capabilities of the learning phase than standard training using only input-output examples.
Summary
This paper presents differentiable Neural Computational Machines (∂NCM), an abstraction of existing neural abstract machines such as Neural Turing Machines (NTMs) and Neural Random Access Machines (NRAMs). Using this abstraction, the paper proposes loss terms for incorporating supervision on execution traces. Adding supervision on execution traces in ∂NCM improves performance over NTM and NRAM which are trained end-to-end from input/output examples only. The observation that adding additional forms of supervision through execution traces improves generalization may be unsurprising, but from what I understand the main contribution of this paper lies in the abstraction of existing neural abstract machines to ∂NCM. However, this abstraction does not seem to be particularly useful for defining additional losses based on trace information. Despite the generic subtrace loss (Eq 8), there is no shared interface between ∂NCM versions of NTM and NRAM that would allow one to reuse the same subtrace loss in both cases. The different subtrace losses used for NTM and NRAM (Eq 9-11) require detailed knowledge of the underlying components of NTM and NRAM (write vector, tape, register etc.), which questions the value of ∂NCM as an abstraction.
Weaknesses
As explained in the summary, it is not clear to me why the abstraction to NCM is useful if one still needs to define specific subtrace losses for different neural abstract machines.
The approach seems to be very susceptible to the weight of the subtrace loss λ, at least when training NTMs. In my understanding each of the trace supervision information (hints, e.g. the ones listed in Appendix F) provides a sensible inductive bias we would the NTM to incorporate. Are there instances where these biases are noisy, and if not, could we incorporate all of them at the same time despite the susceptibility w.r.t λ?
NTMs and other recent neural abstract machines are often tested on rather toyish algorithmic tasks. I have the impression providing extra supervision in form of execution traces makes these tasks even more toyish. For instance, when providing input-output examples as well as the auxiliary loss in Eq6, what exactly is left to learn? What I like about Neural-Programmer Interpreters and Neural Programmer [1] is that they are tested on less toyish tasks (a computer vision and a question answering task respectively), and I believe the presented method would be more convincing for a more realistic downstream task where hints are noisy (as mentioned on page 5).
Minor Comments
p1: Why is Grefenstette et al. (2015) an extension of NTMs or NRAMs? While they took inspiration from NTMs, their Neural Stack has not much resemblance with this architecture.
p2: What is B exactly? It would be good to give a concrete example at this point. I have the feeling it might even be better to explain NCMs in terms of the communication between κ, π and M first, so starting with what I, O, C, B, Q are before explaining what κ and π are (this is done well for NTM as ∂NCM in the table on page 4). In addition, I think it might be better to explain the Controller before the Processor. Furthermore, Figure 2a should be referenced in the text here.
p4 Eq3: There are two things confusing in these equations. First, w is used as the write vector here, whereas on page 3 this is a weight of the neural network. Secondly, π and κ are defined on page 2 as having an element from W as first argument, which are suddenly omitted on page 4.
p4: The table for NRAM as ∂NCM needs a bit more explanation. Where does {1}=I come from? This is not obvious from Appendix B either.
p3 Fig2/p4 Eq4: Related to the concern regarding the usefulness of the ∂NCM abstraction: While I see how NTMs fit into the NCM abstraction, this is not obvious at all for NRAMs, particularly since in Fig 2c modules are introduced that do not follow the color scheme of κ and π in Fig 2a (ct, at, bt and the registers).
p5: There is related work for incorporating trace supervision into a neural abstract machine that is otherwise trained end-to-end from input-output examples [2].
p5: "loss on example of difficulties" -> "loss on examples of the same difficulty"
p5: Do you have an example for a task and hints from a noisy source?
Citation style: sometimes citation should be in brackets, for example "(Graves et al. 2016)" instead of "Graves et al. (2016)" in the first paragraph of the introduction.
[1] Neelakantan et al. Neural programmer: Inducing latent programs with gradient descent. ICLR. 2015. 
[2] Bosnjak et al. Programming with a Differentiable Forth Interpreter. ICML. 2017.