Evolution strategies or zeroth-order optimization algorithms have become popular in some areas of optimization and machine learning where only the oracle of function value evaluations is available. The central idea in the design of the algorithms is by querying function values of some perturbed points in the neighborhood of the current update and constructing a pseudo-gradient using the function values. In recent years, there is a growing interest in developing new ways of perturbation. Though the new perturbation methods are well motivating, most of them are criticized for lack of convergence guarantees even when the underlying function is convex. Perhaps the only methods that enjoy convergence guarantees are the ones that sample the perturbed points uniformly from a unit sphere or from a multivariate Gaussian distribution with an isotropic covariance. In this work, we tackle the non-convergence issue and propose sampling perturbed points from a mixture of distributions. Experiments show that our proposed method can identify the best perturbation scheme for the convergence and might also help to leverage the complementariness of different perturbation schemes.
This paper proposes to change the sampling scheme of evolutionary strategies to sample from a mixture of distributions (instead of the standard way of sampling from one Gaussian).
Although this seems to provide slightly better performance, the contribution is incremental and more importantly the paper lacks clarity, especially regarding the theory part. The experiments are also not on par with what’s typically expected in the literature. Overall, this paper is not ready for acceptance at a venue such as ICLR.
EXP3.P algorithm
This is one of the key components of the algorithm presented by the authors but it is not properly explained in the paper. In fact, the algorithm implemented by the authors is very unclear to me. Below I quote the description given by the authors:
“One can view that EXP3.P runs synchronously with Algorithm 5 in the way that step 4 of EXP3.P implements step 5 of Algorithm 5 and that step 5 and the subsequent steps of EXP3.P are conducted after step 8 of Algorithm 5 is finished“
This is a very poor description of the algorithm. The authors should give a clear pseudo-code to explain the algorithm.
Theoretical guarantees
1) Again, the paper is very unclear on what algorithm is being implemented. The authors claim they can directly use the guarantees provided by Bubeck et al but this is unclear to me given that the algorithm is never fully stated. This needs to be clarified.
2) The authors claim asymptotic convergence, specifically they claim “Therefore, one might show that the proposed algorithm converges with a high probability.“. This proof should be provided in the paper, not left as an exercise to the reader...
3) Does Theorem 2 (Theorem 3.3 from Bubeck) require convexity of the function? Can you comment on guarantees for non-convex functions?
Prior work
The authors do not give a clear description of prior work. There is a large literature on proving convergence guarantees under weaker assumptions than discussed in the paper, see e.g.
Y. Diouane, S. Gratton, and L. N. Vicente. Globally convergent evolution strategies. Math.
Program., 152:467–490, 2015.
Y. Diouane, S. Gratton, and L. N. Vicente. Globally convergent evolution strategies for
constrained optimization. Comput. Optim. Appl., 62:323–346, 2015.
Vincente. Worst case complexity of direct search
Experiments
This is also a weak part of the paper.
1) The authors only run experiments on very small datasets. Regarding the non-convex task, it would be much more interesting to consider a reinforcement learning task where evolutionary methods typically perform well.
2) You should include stronger baselines too, e.g. https://arxiv.org/abs/1806.10230.
3) An interesting question to look at would be to analyze the behavior of the algorithm as a function of the number of mixture components. Also, what is the computational increase in terms of the number of components? Consider giving plots in terms of computation time.