Many applications in machine learning require optimizing a function whose true gradient is unknown, but where surrogate gradient information (directions that may be correlated with, but not necessarily identical to, the true gradient) is available instead. This arises when an approximate gradient is easier to compute than the full gradient (e.g. in meta-learning or unrolled optimization), or when a true gradient is intractable and is replaced with a surrogate (e.g. in certain reinforcement learning applications or training networks with discrete variables). We propose Guided Evolutionary Strategies, a method for optimally using surrogate gradient directions along with random search. We define a search distribution for evolutionary strategies that is elongated along a subspace spanned by the surrogate gradients. This allows us to estimate a descent direction which can then be passed to a first-order optimizer. We analytically and numerically characterize the tradeoffs that result from tuning how strongly the search distribution is stretched along the guiding subspace, and use this to derive a setting of the hyperparameters that works well across problems. Finally, we apply our method to example problems including truncated unrolled optimization and training neural networks with discrete variables, demonstrating improvement over both standard evolutionary strategies and first-order methods (that directly follow the surrogate gradient). We provide a demo of Guided ES at: redacted URL
Summary: The paper proposes a method to improve random search by building a subspace of the previous k surrogate gradients, mixing it with an isotropic Gaussian distribution to improve the search. Results reported shows are good compared to other approaches for learning weights of neural networks. However, paper lacks clarity and is quite hard to follow.
Quality: The paper presents a well-designed approach that is able to deal with optimization in high dimensionality space, by building a lower order surrogate model build upon the previous gradients computed with this surrogate model. The analysis appears to be correct and provide credential to the approach. Results reported are very good. However, testing is relatively limited to few cases. More experimental results on a good set of problems with several methods would have made the paper stronger and more convincing.
Clarity: The paper is hard to follow. Maybe because I am not completely familiar with the topic, but many elements presented lacks some context. The authors appear clearly to be knowledgeable of their topics, but lacks the capacity to provide all required background to follow their thoughts. The method could have been better illustrated, I found that Fig. 1a not enough to explain the method, while Fig. 1b and other training curves not useful to understand the approach. Some pseudo-code to illustrate the use of the proposed method might certainly help to improve clarity. Sec. 3.2 is not enough to understand well the approach.
Originality: The approach is allowing a nice trade-off between pure random search and guide search through a surrogate model over a subspace of limited dimensionality. This is in-line of some work on the use of ES for training neural networks, but I am not aware of other similar work although I am not super knowledgeable of the field. 
Significance: The approach can have its impact for optimizing deep networks with no gradient, but more exhaustive experimental testing would be required.
Pros and cons:
+ Sound approach
+ Good theoretical support of the approach (bias-variance analysis)
+ Great results reported
+ Of importance for optimizing without gradients
- Presentation of the method lacking many details and not very clear
- Overall quality of the paper is subpar, tend to be very textual and hard to follow in several parts
- Experiments are not exhaustive and detailed. Loss plots are provided for some methods compared. Looks more like a preliminary validation. 
I think that if the paper can be rewritten to be more tight, clearer in its presentation, with figures and pseudo-code to illustrate the method better, with more exhaustive testing, it can be really great. Current, the method appears to be great, but the writing quality of the paper is not yet there.