A Generalization Result for Convergence in Learning-to-Optimize
<|reference_start|>A Generalization Result for Convergence in Learning-to-Optimize: Convergence in learning-to-optimize is hardly studied, because conventional convergence guarantees in optimization are based on geometric arguments, which cannot be applied easily to learned algorithms. Thus, we develop a probabilistic framework that resembles deterministic optimization and allows for transferring geometric arguments into learning-to-optimize. Our main theorem is a generalization result for parametric classes of potentially non-smooth, non-convex loss functions and establishes the convergence of learned optimization algorithms to stationary points with high probability. This can be seen as a statistical counterpart to the use of geometric safeguards to ensure convergence. To the best of our knowledge, we are the first to prove convergence of optimization algorithms in such a probabilistic framework.<|reference_end|>
@article{sucker2024a,
    title={A Generalization Result for Convergence in Learning-to-Optimize},
    author={Michael Sucker and Peter Ochs},
    journal={arXiv preprint arXiv:2410.07704},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.07704},
    primaryClass={cs.LG math.OC math.PR}
}