Zeroth-Order Stochastic Variance Reduction for Nonconvex Optimization
As application demands for zeroth-order (gradient-free) optimization accelerate, the need for variance reduced and faster converging approaches is also intensifying. This paper addresses these challenges by presenting: a) a comprehensive theoretical analysis of variance reduced zeroth-order (ZO) optimization, b) a novel variance reduced ZO algorithm, called ZO-SVRG, and c) an experimental evaluation of our approach in the context of two compelling applications, black-box chemical material classification and generation of adversarial examples from black-box deep neural network models. Our theoretical analysis uncovers an essential difficulty in the analysis of ZO-SVRG: the unbiased assumption on gradient estimates no longer holds. We prove that compared to its first-order counterpart, ZO-SVRG with a two-point random gradient estimator could suffer an additional error of order O(1/b), where b is the mini-batch size. To mitigate this error, we propose two accelerated versions of ZO-SVRG utilizing variance reduced gradient estimators, which achieve the best rate known for ZO stochastic optimization (in terms of iterations). Our extensive experimental results show that our approaches outperform other state-of-the-art ZO algorithms, and strike a balance between the convergence rate and the function query complexity.
This paper extensively investigates the advantage of variance reduction for stochastic zeroth-order in non-convex optimization (where one does not have access to the gradient of the objective). They consider three different approaches to zo gradient estimation, including  RandGradEst, Avg-RandGradEst, and CoordGradEst. For each method, they propose a convergence guarantee. The main contribution (expressed in Theorem 3) is the derivation of the improved rate O(1/T) for CoordGradEst using variance reduction. The result is novel and can potentially have a high impact on non-convex optimization.   Although the authors have done a good work to provide a rich theoretical result, I think that the presentation of the paper can be improved a lot. Here, I’ve listed my concerns regarding results and the presentation:   1. My first concern is the dependency of all convergence result to the number of samples n. We can not hide this factor in the O(.) notation because it can crucially change the total time complexity of algorithms (see the seminal paper “Stochastic Variance Reduction for Nonconvex Optimization” where the number of samples is clearly presented in time complexity). 2. The paper uses existing proof techniques of stochastic non-convex optimization and variance reduction without citing them. For example, the Lyapunov function of Eq. 104 is previously used in “Stochastic Variance Reduction for Nonconvex Optimization”. 3. Although the authors have tried to highlight the difference between the difference between different 0-order gradient estimates, I am not convinced that CoordGradEst is the best approach with such a significant improvement in the convergence. The empirical results (in the experiments section) also show that Avg-RandGradEst outperforms as well as CoordGradEst. I think that the source of the convergence gap is the loose bound of Eq 37. If one can prove that \had{\nabla} f_i is Lipschitz in expectation then a same converge rate O(1/T) can be driven for  Avg-RandGradEst and even RandGradEst.  4.  The proof technique behind all three theorems is the same. I highly recommend to integrate the similar parts of proof in lemmas and propositions instead of repeating all the derivations.  If authors represent the upper-bound of proposition 1 for all three 0-order gradient estimates, then reads can understand better the difference between these three approaches. I think that the last term of upper bound vanishes for CoordGradEst and this the source of the improved rate (still I think that this bound can be improved for RandGradEst). Regarding the bound of Eq 99, I could not reach the same upper bound using Eqs (34)-(36).  5. I recommend to first analyze the effect of variance reduction on the smoothed function f_\mu instead of the original function f. Since 0-order gradient estimate provides an unbiased estimate on f_\mu, this helps to simplify the analysis. Then we can relate f_\mu to f in terms of suboptimality. Furthermore, the objective f_\mu as a smooth version of the main function might be more favorable for non-convex optimization.  --------------------------------------------- I have read the author response. They addressed most of my concerns, hence I've updated my score. Regarding the 3rd comment, If they assume that stochastic gradients are Lipschitz, they might be able to achieve better bound in Eq. 37.