Stability and Performance Analysis of Discrete-Time ReLU Recurrent Neural Networks
<|reference_start|>Stability and Performance Analysis of Discrete-Time ReLU Recurrent Neural Networks: This paper presents sufficient conditions for the stability and $\ell_2$-gain performance of recurrent neural networks (RNNs) with ReLU activation functions. These conditions are derived by combining Lyapunov/dissipativity theory with Quadratic Constraints (QCs) satisfied by repeated ReLUs. We write a general class of QCs for repeated RELUs using known properties for the scalar ReLU. Our stability and performance condition uses these QCs along with a "lifted" representation for the ReLU RNN. We show that the positive homogeneity property satisfied by a scalar ReLU does not expand the class of QCs for the repeated ReLU. We present examples to demonstrate the stability / performance condition and study the effect of the lifting horizon.<|reference_end|>
@article{noori2024stability,
    title={Stability and Performance Analysis of Discrete-Time ReLU Recurrent
  Neural Networks},
    author={Sahel Vahedi Noori, Bin Hu, Geir Dullerud, and Peter Seiler},
    journal={arXiv preprint arXiv:2405.05236},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.05236},
    primaryClass={eess.SY cs.LG cs.SY math.OC}
}