One related work that came to mind was Neural Turing Machines, which augments the LSTM network with a (fixed size) memory [1]. I did see a brief reference to the Graves et al. paper in the paper linked to in the original post, but I&#x27;m also wondering if someone more knowledgeable could elaborate on the difference.<p>My own guess would be that the models are similar (both using NNs with external memory, trained through backprop), but this has a simpler memory structure and so could be trained more easily. It seems like the choice of regular RNNs vs. an LSTM network doesn&#x27;t matter much since the Graves paper also compared LSTM vs. feedforward controllers and the results were about the same, with the feedforward controllers actually learning a bit faster. The key of both works seems to be in making a continuous system that can be trained to use external memory through backprop.<p>[1] <a href="http:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;1410.5401" rel="nofollow">http:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;1410.5401</a>