Metric-Entropy Limits on Nonlinear Dynamical System Learning
<|reference_start|>Metric-Entropy Limits on Nonlinear Dynamical System Learning: This paper is concerned with the fundamental limits of nonlinear dynamical system learning from input-output traces. Specifically, we show that recurrent neural networks (RNNs) are capable of learning nonlinear systems that satisfy a Lipschitz property and forget past inputs fast enough in a metric-entropy optimal manner. As the sets of sequence-to-sequence maps realized by the dynamical systems we consider are significantly more massive than function classes generally considered in deep neural network approximation theory, a refined metric-entropy characterization is needed, namely in terms of order, type, and generalized dimension. We compute these quantities for the classes of exponentially-decaying and polynomially-decaying Lipschitz fading-memory systems and show that RNNs can achieve them.<|reference_end|>
@article{pan2024metric-entropy,
    title={Metric-Entropy Limits on Nonlinear Dynamical System Learning},
    author={Yang Pan, Clemens Hutter, Helmut B"olcskei},
    journal={arXiv preprint arXiv:2407.01250},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.01250},
    primaryClass={cs.LG cs.IT math.DS math.IT}
}