Unifying back-propagation and forward-forward algorithms through model predictive control
<|reference_start|>Unifying back-propagation and forward-forward algorithms through model predictive control: We introduce a Model Predictive Control (MPC) framework for training deep neural networks, systematically unifying the Back-Propagation (BP) and Forward-Forward (FF) algorithms. At the same time, it gives rise to a range of intermediate training algorithms with varying look-forward horizons, leading to a performance-efficiency trade-off. We perform a precise analysis of this trade-off on a deep linear network, where the qualitative conclusions carry over to general networks. Based on our analysis, we propose a principled method to choose the optimization horizon based on given objectives and model specifications. Numerical results on various models and tasks demonstrate the versatility of our method.<|reference_end|>
@article{ren2024unifying,
    title={Unifying back-propagation and forward-forward algorithms through model
  predictive control},
    author={Lianhai Ren, Qianxiao Li},
    journal={arXiv preprint arXiv:2409.19561},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.19561},
    primaryClass={cs.LG math.OC}
}