Global Convergence of Decentralized Retraction-Free Optimization on the Stiefel Manifold
<|reference_start|>Global Convergence of Decentralized Retraction-Free Optimization on the Stiefel Manifold: Many classical and modern machine learning algorithms require solving optimization tasks under orthogonal constraints. Solving these tasks often require calculating retraction-based gradient descent updates on the corresponding Riemannian manifold, which can be computationally expensive. Recently Ablin et al. proposed an infeasible retraction-free algorithm, which is significantly more efficient. In this paper, we study the decentralized non-convex optimization task over a network of agents on the Stiefel manifold with retraction-free updates. We propose \textbf{D}ecentralized \textbf{R}etraction-\textbf{F}ree \textbf{G}radient \textbf{T}racking (DRFGT) algorithm, and show that DRFGT exhibits ergodic $\mathcal{O}(1/K)$ convergence rate, the same rate of convergence as the centralized, retraction-based methods. We also provide numerical experiments demonstrating that DRFGT performs on par with the state-of-the-art retraction based methods with substantially reduced computational overhead.<|reference_end|>
@article{sun2024global,
    title={Global Convergence of Decentralized Retraction-Free Optimization on the
  Stiefel Manifold},
    author={Youbang Sun, Shixiang Chen, Alfredo Garcia, Shahin Shahrampour},
    journal={arXiv preprint arXiv:2405.11590},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.11590},
    primaryClass={cs.LG math.OC}
}