Optimization over bounded-rank matrices through a desingularization enables joint global and local guarantees
<|reference_start|>Optimization over bounded-rank matrices through a desingularization enables joint global and local guarantees: Convergence guarantees for optimization over bounded-rank matrices are delicate to obtain because the feasible set is a non-smooth and non-convex algebraic variety. Existing techniques include projected gradient descent, fixed-rank optimization (over the maximal-rank stratum), and the LR parameterization. They all lack either global guarantees (the ability to accumulate only at critical points) or fast local convergence (e.g., if the limit has non-maximal rank). We seek optimization algorithms that enjoy both.   Khrulkov and Oseledets [2018] parameterize the bounded-rank variety via a desingularization to recast the optimization problem onto a smooth manifold. Building on their ideas, we develop a Riemannian geometry for this desingularization, also with care for numerical considerations. We use it to secure global convergence to critical points with fast local rates, for a large range of algorithms. On matrix completion tasks, we find that this approach is comparable to others, while enjoying better general-purpose theoretical guarantees.<|reference_end|>
@article{rebjock2024optimization,
    title={Optimization over bounded-rank matrices through a desingularization
  enables joint global and local guarantees},
    author={Quentin Rebjock and Nicolas Boumal},
    journal={arXiv preprint arXiv:2406.14211},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.14211},
    primaryClass={math.OC cs.NA math.NA}
}