INITIALIZED EQUILIBRIUM PROPAGATION FOR BACKPROP-FREE TRAINING
Deep neural networks are almost universally trained with reverse-mode automatic differentiation (a.k.a. backpropagation). Biological networks, on the other hand, appear to lack any mechanism for sending gradients back to their input neurons, and thus cannot be learning in this way. In response to this, Scellier & Bengio (2017) proposed Equilibrium Propagation -a method for gradient-based training of neural networks which uses only local learning rules and, crucially, does not rely on neurons having a mechanism for back-propagating an error gradient. Equilibrium propagation, however, has a major practical limitation: inference involves doing an iterative optimization of neural activations to find a fixed-point, and the number of steps required to closely approximate this fixed point scales poorly with the depth of the network. In response to this problem, we propose Initialized Equilibrium Propagation, which trains a feedforward network to initialize the iterative inference procedure for Equilibrium propagation. This feed-forward network learns to approximate the state of the fixed-point using a local learning rule. After training, we can simply use this initializing network for inference, resulting in a learned feedforward network. Our experiments show that this network appears to work as well or better than the original version of Equilibrium propagation while requiring fewer steps to converge. This shows how we might go about training deep networks without using backpropagation.
This paper presents an improvement on the local/derivative-free learning algorithm equilibrium propagation. Specifically, it trains a feedforward network to initialize the iterative optimization process in equilibrium prop, leading to greater stability and computational efficiency, and providing a network that can later be used for fast feedforward predictions on test data. Non-local gradient terms are dropped when training the feedforward network, so that the entire system still doesn't require backprop. There is a neat theoretical result showing that, in the neighborhood of the optimum, the dropped non-local gradient terms will be correlated with the retained gradient terms.
My biggest concern with this paper is the lack of significant literature review, and that it is not placed in the context of previous work. There are only 12 references, 5 of which come from a single lab, and almost all of which are to extremely recent papers. Before acceptance, I would ask the authors to perform a literature search, update their paper to include citations to and discussion of previous work, and better motivate the novelty of their paper relative to previous work. Luckily, this is a concern that is addressable during the rebuttal process! If the authors perform a literature search, and update their paper appropriately, I will raise my score as high as 7.
Here are a few related topic areas which are currently not discussed in the paper. *I am including these as a starting point only! It is your job to do a careful literature search. I am completely sure there are obvious connections I'm missing, but these should provide some entry points into the citation web.*
- The "method of auxiliary coordinates" introduces soft (often quadratic) couplings between post- and pre- activations in adjacent layers which, like your distributed quadratic penalty, eliminate backprop across the couplings. I believe researchers have also done similar things with augmented Lagrangian methods. A similar layer-local quadratic penalty also appears in ladder networks.
- Positive/negative phase (clamped / unclamped phase) training is ubiquitous in energy based models. Note though that it isn't used in classical Hopfield networks. You might want to include references to other work in energy based models for both this and other reasons. e.g., there may be some similarities between this approach and continuous-valued Boltzmann machines?
- In addition to feedback alignment, there are other approaches to training deep neural networks without standard backprop. examples include: synthetic gradients, meta-learned local update rules, direct feedback alignment, deep Boltzmann machines, ...
- There is extensive literature on biologically plausible learning rules -- it is a field of study in its own right. As the paper is motivated in terms of biological plausibility, it would be good to include more general context on the different approaches taken to biological plausibility.
More detailed comments follow:
Thank you for including the glossary of symbols!
"Continuous Hopfield Network" use lowercase for this (unless introducing acronym)
"is the set non-input" -> "is the set of non-input"
"$\alpha = ...$ ... $\alpha_j \subset ...$" I could not make sense of the set notation here.
would recommend using something other than rho for nonlinearity. rho is rarely used as a function, so the prior of many readers will be to interpret this as a scalar. phi( ) or f( ) or h( ) are often used as NN nonlinearities.
inline equation after "clamping factor" -- believe this should just be C, rather than \partial C / \partial s.
Move definition of \mathcal O up to where the symbol is first used.
text before eq. 7 -- why train to approximate s- rather than s+? It seems like s+ would lead to higher accuracy when this is eventually used for inference.
eq. 10 -- doesn't the regularization term also decrease the expressivity of the Hopfield network? e.g. it can no longer engage in "explaining away" or enforce top-down consistency, both of which are powerful positive attributes of iterative estimation procedures.
notation nit: it's confusing to use a dot to indicate matrix multiplication. It is commonly used in ML to indicate an inner product between two vectors of the same shape/orientation. Typically matrix multiplication is implied whenever an operator isn't specified (eg x w_1 is matrix multiplication).
eq. 12 -- is f' supposed to be h'? And wasn't the nonlinearity earlier introduced as rho? Should settle on one symbol for the nonlinearity.
This result is very cool. It only holds in the neighborhood of the optimum though. At initialization, I believe the expected correlation is zero by symmetry arguments (eg, d L_2 / d s_2 is equally likely to have either sign). Should include an explicit discussion of when this relationship is expected to hold.
"proportional to" -> "correlated with" (it's not proportional to)
sec. 3 -- describe nonlinearity as "hard sigmoid"
beta is drawn from uniform distribution including negative numbers? beta was earlier defined to be positive only.
Figure 2 -- how does the final achieved test error change with the number of negative-phase steps? ie, is the final classification test error better even for init eq prop in the bottom row than it is in the top?
The idea of initializing an iterative settling process with a forward pass goes back much farther than this. A couple contexts being deep Boltzmann machines, and the use of variational inference to initialize Monte Carlo chains
sect 4.3 -- "the the" -> "to the"