Learning-Rate-Free Stochastic Optimization over Riemannian Manifolds
<|reference_start|>Learning-Rate-Free Stochastic Optimization over Riemannian Manifolds: In recent years, interest in gradient-based optimization over Riemannian manifolds has surged. However, a significant challenge lies in the reliance on hyperparameters, especially the learning rate, which requires meticulous tuning by practitioners to ensure convergence at a suitable rate. In this work, we introduce innovative learning-rate-free algorithms for stochastic optimization over Riemannian manifolds, eliminating the need for hand-tuning and providing a more robust and user-friendly approach. We establish high probability convergence guarantees that are optimal, up to logarithmic factors, compared to the best-known optimally tuned rate in the deterministic setting. Our approach is validated through numerical experiments, demonstrating competitive performance against learning-rate-dependent algorithms.<|reference_end|>
@article{dodd2024learning-rate-free,
    title={Learning-Rate-Free Stochastic Optimization over Riemannian Manifolds},
    author={Daniel Dodd, Louis Sharrock, Christopher Nemeth},
    journal={arXiv preprint arXiv:2406.02296},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.02296},
    primaryClass={cs.LG math.OC}
}