A Forward Model at Purkinje Cell Synapses Facilitates Cerebellar Anticipatory Control
How does our motor system solve the problem of anticipatory control in spite of a wide spectrum of response dynamics from different musculo-skeletal systems, transport delays as well as response latencies throughout the central nervous system? To a great extent, our highly-skilled motor responses are a result of a reactive feedback system, originating in the brain-stem and spinal cord, combined with a feed-forward anticipatory system, that is adaptively fine-tuned by sensory experience and originates in the cerebellum. Based on that interaction we design the counterfactual predictive control (CFPC) architecture, an anticipatory adaptive motor control scheme in which a feed-forward module, based on the cerebellum, steers an error feedback controller with counterfactual error signals. Those are signals that trigger reactions as actual errors would, but that do not code for any current or forthcoming errors. In order to determine the optimal learning strategy, we derive a novel learning rule for the feed-forward module that involves an eligibility trace and operates at the synaptic level. In particular, our eligibility trace provides a mechanism beyond co-incidence detection in that it convolves a history of prior synaptic inputs with error signals. In the context of cerebellar physiology, this solution implies that Purkinje cell synapses should generate eligibility traces using a forward model of the system being controlled. From an engineering perspective, CFPC provides a general-purpose anticipatory control architecture equipped with a learning rule that exploits the full dynamics of the closed-loop system.
The authors extend the classic cerebellar model of Marr-Albus (1969). By doing so they also provide a framework to obtain an optimal eligibility trace. Their contribution is important in understanding the cerebellar function, but also may find practical applications in control-system problems. I would like to make a few comments regarding the exposition of their solution and its context with respect to previous works. 
1) As a person from outside the control theory field, I found it difficult to follow some of the derivations and arguments in the manuscript. In particular, the authors might explain in more detail the meaning of matrices A, B, C and D on page 4 (perhaps also by referring back to them in Figure 2). They should also present the form of these matrices used in the simulations in section 3.4. Similarly, the quantities k_p and k_i on page 6 are not defined. They might be trivial for control theorist, but not for readers outside the field.  2) From what I understood, the eligibility trace (h_j on page 5) is computed from the known system transition matrix and cerebellar basis. However, in realistic biological setup this matrix will not be a priori known. How will the eligibility trace be estimated in the cerebellum? Is this matrix hard-coded in the cerebellar architecture? If so, how is it adapted to new tasks? If it is learnt, what would the learning rule be?  3) The framework proposed by the authors has some similarities with predictive coding proposed for vision (Rao and Ballard, Nat Neurosci, 1999). Could the authors expose the similarities and differences? One key difference seems to be that in the authors' approach the error signal e[n] is not used to compute the predicted output o[n], but only to adjust the weights. How would the authors motivate this choice?  4) If the goal of the study is to find a realistic model of cerebellum, the authors should discuss their results in context of experimental findings about cerebellum architecture and function. For example, human study performed by Blakmore  et al. (NeuroReport, 2001) suggest that the cerebellum encodes the error signal. Is this finding consistent with what the authors propose? If not, could their model be used to explain the data?  Minor comments:  line 4: extend -> extent line 164: with relation -> with respect to line 165: get derive -> derive line 170: different -> difference line 177: e[k,n] -> did the authors mean e[k - n]? line 181: "has been computing convolving" - not clear what is meant, please correct line 202: subtracted to the current weight -> subtracted from the current weight line 220: meaning of symbols k_p and k_i is not clear line 226: why these values? what does "spaced by" mean? Is this spacing between tuning curves of different fibers? line 248: are crucial for not only that but also to account -> are not only crucial for [the delays?] but also they account line 304: what predictions precisely do the authors mean?