Zeroth-order optimization is the process of minimizing an objective f (x), given oracle access to evaluations at adaptively chosen inputs x. In this paper, we present two simple yet powerful GradientLess Descent (GLD) algorithms that do not rely on an underlying gradient estimate and are numerically stable. We analyze our algorithm from a novel geometric perspective and present a novel analysis that shows convergence within an -ball of the optimum in O(kQ log(n) log(R/ )) evaluations, for any monotone transform of a smooth and strongly convex objective with latent dimension k < n, where the input dimension is n, R is the diameter of the input space and Q is the condition number. Our rates are the first of its kind to be both 1) poly-logarithmically dependent on dimensionality and 2) invariant under monotone transformations. We further leverage our geometric perspective to show that our analysis is optimal. Both monotone invariance and its ability to utilize a low latent dimensionality are key to the empirical success of our algorithms, as demonstrated on BBOB and MuJoCo benchmarks.
Update after rebuttal: I found the rebuttal convincing and I liked the fact that concerns regarding empirical justification were addressed. Consequently, I increase my score from "Weak Reject" to "Weak Accept".
--------------------------
This paper focuses on derivative-free, or zero-th order, optimization. That is the setting where a function may be continuous and/or smooth and/or (quasi)-convex, however, we do not have access to the gradients. As such, it is not possible to apply standard gradient descent methods. The paper proposes an algorithm for "gradientless" descend. The basic intuition is that by careful randomly sampling it is quite probable that a lower objective value will be attained. Doing so recursively can then lead to the optimum with high probability. Clearly, in such a setting it is quite important to clarify what is "careful random sampling". To this end the paper casts this as sampling from a Gaussian ball of specific radius, chosen such that the samples are with high probality below the current level set (the hyperplance of equivalent solutions f(x) as our current solution f(x_t)). The paper derives and proves various theorems on how to select the optimal radius and how to perform the sampling. Specifically, the case of strongly convex and smooth functions is analyzed, however, the paper also shows how this generalizes to functions after a monotone transformation (thus leading to quasi-convex functions) and with extra error perturbations. The proposed algorithm is compared on a synthetic experiment, and a selection of MuJoCo benchmarks.
Strengths:
+ The derivations and the theorems are non-trivial. There is some serious analysis regarding the selection of radius of Gaussian balls. I would like to congratulate the authors for this. I particularly like the extension to having a perturbing function h(x), leading to a more realistic setup.
+ I also particularly like that the algorithm is able to recover subspaces automatically. This definitely makes the algorithm much more practical and more efficient.
+ The writing and the presentation are rather clear and well taken care of. Although there are several theorems, it was not too hard to follow the flow of the paper. The algorithm boxes are also concise and clear, helping with understanding the final result.
Weaknesses:
+ Although the contributions of the work are mostly on the theoretical side, I have a hard time grasping how useful is the algorithm in practice. For one, there is the assumption of strongly convex and smooth function. Granted, there is the relaxed case of having the perturbing function h(x), howevrer, in that case it seems that the algorithm becomes a slower by an order of 60 δ k Q_g(A). How fast or slow is this in practice? Even with applying the monotone function, the algorithm becomes more practical by being applicable to quasi-convex setups. However, how realistic is that a function will in practice be strictly monotone? While most of this may be hard to be theoretically proven, they can be experimentally tested.
+ Also, given that the paper is interested in black box functions, we cannot have much information regarding the function. So, what happens when the function is not strongly convex or not always smooth? Furthermore, how realistic is to know the condition number, that is the maximum derivative (or an upper bound of it) since we do not have access to the gradients in the first place?
+ I would say that the paper could benefit from a more extensive experimental section. Currently only a single synthetic function is analyzed, also under a single monotone exponential transformation. From a more practical point of view, MuJoCo environments are also examined. However, there exist no comparisons with other methods in the literature, including ARS. Another relevant algorithm to compare with would be the stochastic tree points (Bergou et al., 2019), if not experimentally at least theoretically. In the end, it quite unclear whether the algorithm work well in practice. Some experiments that could shed light would relate to how sensitive the algorithm is to the convexity/smoothness assumptions, how sensitive the algorithm is to the perturbing function h(x), how sensitive is the algorithm to the present of a lower-dimensional subspace that needs to be discovered. And for the MuJoCo experiments, the algorithm can compare at least with ARS.
+ In the experiments it seems the paper is particularly good in high dimensions. Can this be more precisely connected to the derived theory in the discussion of the experiments? Does this relate to the better subspaces k that are discovered automatically by the algorithm?
+ It is unclear how many evaluations are needed per step, that is what is the K value in the algorithm box? Also, there are at least two balls to sample from, so twice as many evaluations, correct?
+ It is unclear why Bayesian Optimization is not considered for at least comparing experimentally. Currently, the paper discards them on the grounds that they do not provide strong theoretically guarantees. However, it would be interesting to examine at least in practice how good/bad are these algorithms in comparison to the proposed one. Two recent bayesian optimization papers that can be considered for continuous and discrete inputs are
[1] BOCK: Bayesian Optimization with Cylindrical Kernels, C. Oh, E. Gavves, M. Welling, ICML 2018
[2] BOCS: Bayesian Optimization of Combinatorial Structures, R. Baptista, M. Poloczek, ICML 2018
+ The paper does not have a conclusion. That shows great sloppiness. Also, i/n the abstract, do you mean to say k>=n or k<=n?
To conclude, I recommend weak rejection only because I am not completely convinced by the experiments and do not know if the proposed algorithm is competitive against reasonable baselines and in more complex setups. I am more than happy to upgrade my score if experiments become more clear.