Feedforward Learning of Mixture Models
We develop a biologically-plausible learning rule that provably converges to the class means of general mixture models. This rule generalizes the classical BCM neural rule within a tensor framework, substantially increasing the generality of the learning problem it solves. It achieves this by incorporating triplets of samples from the mixtures, which provides a novel information processing interpretation to spike-timing-dependent plasticity. We provide both proofs of convergence, and a close fit to experimental data on STDP.
Summary:The paper extends the classical BCM learning rule to utilize information from spike triplets.It is shown that the update rule can learn selectivity for mixture distributions (by convergingto the class means). Quality:By employing tensor notation, the paper shows that the BCM rule can be generalized to use information frommore than a pair of spikes (spike triplets are used for the examples). While the model has fewer parametersthan previous learning algorithms based on spike triplets or quadruplets and the method can be shown to havestable points as class means of mixture distributions, the lack of experimental comparisons with other modelsmakes it hard to gauge the incremental contribution of the model.Clarity:The paper is in general easy to follow but the derivation of equations on line 269 and 273 was unclear to me. Originality:STDP utilizing spike triplets has been introduced in the past by Pfister and Gerstner. Also, Froemke and Danpublished a model which uses spike quadruplets. Moreover, it seems to me that under Poisson spiking statisticsand certain choice of parameters, the model from Pfister and Gerstner is equivalent to the model described here.In that light, I believe a clear comparison should be made using simulations of the algorithm with those fromPfister/Gertsner, Froemke, etc. The simulations should clearly demonstrate under what circumstances does thepresented model differ (in the expected sense) and is better than that from Pfister and Gerstner.Significance:The paper extends a classical approach in neural learning (BCM) and demonstrates that the extended approach has broadercapabilities than the original approach. However, clear and detailed quantitative comparison to previous research in the field is missing.[1] Pfister, Jean-Pascal, and Wulfram Gerstner. "Triplets of spikes in a model of spike timing-dependent plasticity." The Journal of neuroscience 26.38 (2006): 9673-9682.Notes:Line 79, Baysian -> BayesianLine 99 \delta_{ij} is the more common notation rather than \delta_i^jLine 106 In the definition of c, subscript i should be added.		 Does the example here have multiple neurons or a single neuron and dimensional data? 		 If the former, m_i, m_{i-1} and d_i should be bold in the equaution. If not, c_i needs		 to be defined as m_{i-1} x d_i (without inner product and bold letters) to clarify its 		 a single neuron.Line 147, 148 I think there should be an expectation in front RHS equations on both linesLine 161 m should be bold in R(m)Line 262 same as aboveLine 351 Missing verbLine 410 "Lower figure" needs to be corrected. The figures are next to each other.