Enforcing balance allows local supervised learning in spiking recurrent networks
To predict sensory inputs or control motor trajectories, the brain must constantly learn temporal dynamics based on error feedback. However, it remains unclear how such supervised learning is implemented in biological neural networks. Learning in recurrent spiking networks is notoriously difficult because local changes in connectivity may have an unpredictable effect on the global dynamics. The most commonly used learning rules, such as temporal back-propagation, are not local and thus not biologically plausible. Furthermore, reproducing the Poisson-like statistics of neural responses requires the use of networks with balanced excitation and inhibition. Such balance is easily destroyed during learning. Using a top-down approach, we show how networks of integrate-and-fire neurons can learn arbitrary linear dynamical systems by feeding back their error as a feed-forward input. The network uses two types of recurrent connections: fast and slow. The fast connections learn to balance excitation and inhibition using a voltage-based plasticity rule. The slow connections are trained to minimize the error feedback using a current-based Hebbian learning rule. Importantly, the balance maintained by fast connections is crucial to ensure that global error signals are available locally in each neuron, in turn resulting in a local learning rule for the slow connections. This demonstrates that spiking networks can learn complex dynamics using purely local learning rules, using E/I balance as the key rather than an additional constraint. The resulting network implements a given function within the predictive coding scheme, with minimal dimensions and activity. The brain constantly predicts relevant sensory inputs or motor trajectories. For example, there is evidence that neural circuits mimic the dynamics of motor effectors using internal models [1]. If the dynamics of the predicted sensory and motor variables change in time, these models may become false [2] and therefore need to be readjusted through learning based on error feedback.
From a modeling perspective, supervised learning in recurrent networks faces many challenges. Earlier models have succeeded in learning useful functions at the cost of non local learning rules that are biologically implausible [3,4]. More recent models based on reservoir computing [5][6][7] transfer the learning from the recurrent network (with now "random", fixed weights) to the readout weights. Using this simple scheme, the network can learn to generate complex patterns. However, the majority of these models use abstract rate units and are yet to be translated into more realistic spiking networks. Moreover, to provide a sufficiently large reservoir, the recurrent network needs to be large, balanced and have a rich and high dimensional dynamics. This typically generates far more activity than strictly required, a redundancy that can be seen as inefficient.
On the other hand, supervised learning models involving spiking neurons have essentially concentrated on the learning of precise spike sequences [8][9][10]. With some exceptions [10,11] these models use feed-forward architectures [12]. In a balanced recurrent network with asynchronous, irregular and highly variable spike trains, such as those found in cortex, the activity has been shown to be 1
chaotic [13,14]. This leads to spike timing being intrinsically unreliable, rendering a representation of the trajectory by precise spike sequences problematic. Moreover, many configurations of spike times may achieve the same goal [15].
Here we derive two local learning rules that drive a network of leaky integrate-and-fire (LIF) neurons into implementing a desired linear dynamical system. The network is trained to minimize the objective x(t) −x(t)
2 + H(r), Wherex(t) is the output of the network decoded from the spikes, x(t) is the desired output, and H(r) is a cost associated with firing (penalizing unnecessary activity, and thus enforcing efficiency). The dynamical system is linear,ẋ = Ax + c, with A being a constant matrix and c a time varying command signal. We first study the learning of an autoencoder, i.e., a network where the desired output is fed to the network as a feedforward input. The autoencoder learns to represent its inputs as precisely as possible in an unsupervised fashion. After learning, each unit represents the encoding error made by the entire network. We then show that the network can learn more complex computations if slower recurrent connections are added to the autoencoder. Thus, it receives the command c along with an error signal and learns to generate the outputx with the desired temporal dynamics. Despite the spike-based nature of the representation and of the plasticity rules, the learning does not enforce precise spike timing trajectories but, on the contrary, enforces irregular and highly variable spike trains.
I greatly appreciate the problem the authors are trying to solve: supervised learning is difficult to implement in a biologically plausible manner, indeed known algorithms (such as backprop through time) are typically non-local. The authors propose a neat solution: unsupervised learning of fast recurrent connections establishes a detailed balance of E and I currents, which is then shown to make relevant aspects of the global output of the network available to each neuron locally. These global-made-local signals can then be used to learn slower connections in order to implement a linear dynamical system.I find the idea really appealing, but I think the results are rather preliminary and the whole study could have been more thorough.- while the spike-based rule used to learn a detailed balance can certainly be shown to achieve its goal in an isolated postsynaptic neuron, it is by no means guaranteed to work in a recurrent network (for just about the same reasons that make local approximations to error backpropagation close to useless in a recurrent net). Why is it that it works here?- learning the slow connections requires very strong feedback (K large). The authors claim that after learning, since the error has vanished anyways, K could be set to zero without affecting performance. Is that true?Is there a figure where we can see that? If the error was exactly zero, then I would buy that, but since it's never perfect, one is allowed to wonder about the stability of the K=0 solution to small errors. Indeed the error can be reduced arbitrarily by making K very large, without even touching the slow weights, so clearly in that case a very small error isn't at all predictive of successful outputs with K=0; for a stable solution to emerge with K small, the error with K large should be further reduced (through learning) by an amount that ought to be made more precise mathematically. I guess what I'm uncomfortable with is that if the authors' claim was true, we could imagine decreasing K progressively as W_slow is being learned, but then at some point we would hit the limit of validity of the first-order expansion of (I-M/K)^{-1} used to derive Eq7 and run into trouble. Am I missing something here?- somehow, the authors postpone the difficult problem of local supervised learning to the (in my opinion, not much easier) learning of identical input, output, and error feedback weights that must all be equal to the same matrix F. Such learning is not addressed here. Could the authors discuss/comment on that? - in their derivations, the authors assume that the inputs to the dynamical system they are trying to learn are constant (or very slow changing). What happens if inputs are faster? (btw, what was the author's choice of input c in the learning setup of Fig3?)  Some minor comments/typos: - line 94: what does it mean for the feedforward weights to "evenly span the input space"?- isn't there a sign issue in Eq2? If W_ij > 0 and V_i > 0, the learning rule is going to strengthen W_ij which makes the imbalance even worse.- l175: adjust to bold, "vector font"- trying to derive Eq.7, I instead got \delta M_{ij} \approx (c_j + Kx_j) e_i. Did the authors drop the constant term on the basis of K being large?- the whole paragraph l.278-l.285 was rather obscure to me; in the end, don't the authors want the error to be fed back through F because of the Fe term in Eq. 9? Couldn't that be stated more simply/clearly?- l221: different from - ref list Gutig - Guetig of Guetig - ref list Deneve - Deneve - caption of fig3B - replace top/bottom by left/right - l387 the the- l403 different fromUPDATE following author's responses: I'm updating my evaluation to 8 due to a satisfying response to my second comment in particular.