Long short-term memory and learning-to-learn in networks of spiking neurons
Recurrent networks of spiking neurons (RSNNs) underlie the astounding computing and learning capabilities of the brain. But computing and learning capabilities of RSNN models have remained poor, at least in comparison with artificial neural networks (ANNs). We address two possible reasons for that. One is that RSNNs in the brain are not randomly connected or designed according to simple rules, and they do not start learning as a tabula rasa network. Rather, RSNNs in the brain were optimized for their tasks through evolution, development, and prior experience. Details of these optimization processes are largely unknown. But their functional contribution can be approximated through powerful optimization methods, such as backpropagation through time (BPTT). A second major mismatch between RSNNs in the brain and models is that the latter only show a small fraction of the dynamics of neurons and synapses in the brain. We include neurons in our RSNN model that reproduce one prominent dynamical process of biological neurons that takes place at the behaviourally relevant time scale of seconds: neuronal adaptation. We denote these networks as LSNNs because of their Long short-term memory. The inclusion of adapting neurons drastically increases the computing and learning capability of RSNNs if they are trained and configured by deep learning (BPTT combined with a rewiring algorithm that optimizes the network architecture). In fact, the computational performance of these RSNNs approaches for the first time that of LSTM networks. In addition RSNNs with adapting neurons can acquire abstract knowledge from prior learning in a Learning-to-Learn (L2L) scheme, and transfer that knowledge in order to learn new but related tasks from very few examples. We demonstrate this for supervised learning and reinforcement learning.
This paper aims to combine deep learning techniques (backpropagation through time and learning to learning) with recurrent networks of spiking neurons, which are more biologically realistic but typically have worse performance and learning capabilities than deep neural networks. They propose a modified version of recurrent spiking neural networks (RSNNs) that includes neural adaptation and BPTT (modified to work with spiking networks), which they term long short-term memory spiking neural networks (LSNNs). They test on two standard benchmark SL tasks: sequential MNIST and TIMIT, and find performance comparable to that of an LSTM. They also test the learning-to-learn capabilities of LSNNs by looking at two task domains: 1) function learning, where the function to be learned is the class of all continuous functions of two variables that can be computed by a 2-layer ANN, as well as simple sine functions, and 2) meta-reinforcement learning in a navigation task, where the goal is placed randomly every episode.  In all, I really enjoyed this paper. It addresses an important disconnect between our understanding of how the brain works, and state of the art deep neural network architectures, which presumably are implementing at least some similar cognitive functions such as visual recognition or sequence processing. The tasks they selected are appropriate, ranging from standard benchmarking tasks in SL to toy tasks which allow for more detailed analysis, and finally a larger scale RL task.   I mainly have a few concerns with regard to clarity and novelty, although I suspect these could be readily addressed in revision.   1. The paper currently feels as if it’s almost 2 separate papers put together, one about LSNNs and one about learning to learn. Are these two contributions completely orthogonal, or is there some interaction? How would LSNNs perform on for example omniglot classification or another benchmark task in which meta-learning is required (perhaps the one used in Santoro et al, 2016 on MANNs)?  2. With regard to clarity, I got confused while reading section 3, because I had expected these to be trained in the “meta-learning” sense, as the previous paragraph (end of section 2) discussed learning-to-learn at length. Going back to my point 1, it’d make much more sense of these tasks were also “meta-learning” tasks. 3. What is the relative importance of the adaptive threshold vs the modified BPTT procedure (with DEEP R) to the performance of the model?  4. It’s important to spell out what are the new contributions made in this paper vs Bellec et al, 2018, which is cited for DEEP R. Without reading this other paper, I can’t tell what is the proposed contribution (was it applied to only FF networks in the other paper?) 5. I’m curious about the disadvantages/advantages of LSNNs vs traditional DNNs. It seems like LSNNs might deal well with sequential data, but what about data without sequential structure, like images? Convnets can capitalize on spatial properties like translational invariance, etc, but would LSNNs always struggle with these? LSTMs are definitely not the first model you’d reach for when classifying MNIST.  Update after reading the authors' rebuttal: While I still think that the paper is pretty strong, I'm pretty disappointed in the response, which never stated what the authors would specifically revise to address the points raised, even with regard to easily addressable issues like clarity. Because of this, I have less confidence that the paper could be revised to be in the top 50% of papers accepted to NIPS. However my opinion is that it should still be accepted.