  Spiking neural networks are a type of artificial neural networks in which
communication between neurons is only made of events, also called spikes. This
property allows neural networks to make asynchronous and sparse computations
and therefore drastically decrease energy consumption when run on specialized
hardware. However, training such networks is known to be difficult, mainly due
to the non-differentiability of the spike activation, which prevents the use of
classical backpropagation. This is because state-of-the-art spiking neural
networks are usually derived from biologically-inspired neuron models, to which
are applied machine learning methods for training. Nowadays, research about
spiking neural networks focuses on the design of training algorithms whose goal
is to obtain networks that compete with their non-spiking version on specific
tasks. In this paper, we attempt the symmetrical approach: we modify the
dynamics of a well-known, easily trainable type of recurrent neural network to
make it event-based. This new RNN cell, called the Spiking Recurrent Cell,
therefore communicates using events, i.e. spikes, while being completely
differentiable. Vanilla backpropagation can thus be used to train any network
made of such RNN cell. We show that this new network can achieve performance
comparable to other types of spiking networks in the MNIST benchmark and its
variants, the Fashion-MNIST and the Neuromorphic-MNIST. Moreover, we show that
this new cell makes the training of deep spiking networks achievable.

Neural and Evolutionary Computing;Machine Learning