Single Point-Based Distributed Zeroth-Order Optimization with a Non-Convex Stochastic Objective Function
<|reference_start|>Single Point-Based Distributed Zeroth-Order Optimization with a Non-Convex Stochastic Objective Function: Zero-order (ZO) optimization is a powerful tool for dealing with realistic constraints. On the other hand, the gradient-tracking (GT) technique proved to be an efficient method for distributed optimization aiming to achieve consensus. However, it is a first-order (FO) method that requires knowledge of the gradient, which is not always possible in practice. In this work, we introduce a zero-order distributed optimization method based on a one-point estimate of the gradient tracking technique. We prove that this new technique converges with a single noisy function query at a time in the non-convex setting. We then establish a convergence rate of $O(\frac{1}{\sqrt[3]{K}})$ after a number of iterations K, which competes with that of $O(\frac{1}{\sqrt[4]{K}})$ of its centralized counterparts. Finally, a numerical example validates our theoretical results.<|reference_end|>
@article{mhanna2024single,
    title={Single Point-Based Distributed Zeroth-Order Optimization with a
  Non-Convex Stochastic Objective Function},
    author={Elissa Mhanna and Mohamad Assaad},
    journal={Proceedings of the 40th International Conference on Machine
  Learning, PMLR 202:24701-24719, 2023},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.05942},
    primaryClass={cs.LG math.OC}
}