SHE2: Stochastic Hamiltonian Exploration and Exploitation for Derivative-Free Optimization
Derivative-free optimization (DFO) using trust region methods is frequently used for machine learning applications, such as (hyper-)parameter optimization without the derivatives of objective functions known. Inspired by the recent work in continuous-time minimizers, our work models the common trust region methods with the exploration-exploitation using a dynamical system coupling a pair of dynamical processes. While the first exploration process searches the minimum of the blackbox function through minimizing a time-evolving surrogation function, another exploitation process updates the surrogation function time-to-time using the points traversed by the exploration process. The efficiency of derivative-free optimization thus depends on ways the two processes couple. In this paper, we propose a novel dynamical system, namely ThePrev---underline{S}tochastic underline{H}amiltonian underline{E}xploration and underline{E}xploitation, that surrogates the subregions of blackbox function using a time-evolving quadratic function, then explores and tracks the minimum of the quadratic functions using a fast-converging Hamiltonian system. The ThePrev algorithm is later provided as a discrete-time numerical approximation to the system. To further accelerate optimization, we present TheName that parallelizes multiple ThePrev threads for concurrent exploration and exploitation. Experiment results based on a wide range of machine learning applications show that TheName outperform a boarder range of derivative-free optimization algorithms with faster convergence speed under the same settings.
 Derivative-free optimization is not a novel domain and your work could benefit from some accepted benchmarking practices. For instance, you can consider the Black-Box Optimization Benchmarking (BBOB) Workshop and its COCO platform which was used to test many optimization algorithms including the ones mentioned in the paper. Benchmarking on BBOB problems would probably reveal that your algorithm fails on non-separable ill-conditioned problems and even on problems like Rosenbrock (e.g., compared to BOBYQA). The results for other algorithms can be downloaded, you don*t need to rerun them. BBOB*s computational budget can be as low as 1 function evaluation. Extended review of Update 17 Nov: I would like to note that I liked the fact that you used several optimization algorithms in your comparison. To my best understanding, several algorithms shown in Figure 3 (e.g., BOBYQA and L-BFGS) would benefit from restarts and it is fairly common to use restarts when the computational budget allows it (it seems to be the case for Figure 3). The results shown in Figure 4 are hard to trust because it does not seem that we observe mean/median results but probably a single run where the results after 1 iteration are drastically different for different algorithms. For instance, after one iteration BOBYQA only tests its second DOE point. Here, again, the issue is that one iteration for BOBYQA is 1 function evaluation while it is several (10) function evaluations for other algorithms. In that scenario, it would be more fair to run BOBYQA with 10 different initializations as well. I don*t understand "Due to the restriction of PyBOBYQA API, we can only provide the function evaluation of the final solution obtained by BOBYQA as a flatline in Figure 4". At the point when your objective function (which is not part of PyBOBYQA API) is called, I suppose you can log everything you need.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
SHE2: Stochastic Hamiltonian Exploration and Exploitation for Derivative-Free Optimization
Derivative-free optimization (DFO) using trust region methods is frequently used for machine learning applications, such as (hyper-)parameter optimization without the derivatives of objective functions known. Inspired by the recent work in continuous-time minimizers, our work models the common trust region methods with the exploration-exploitation using a dynamical system coupling a pair of dynamical processes. While the first exploration process searches the minimum of the blackbox function through minimizing a time-evolving surrogation function, another exploitation process updates the surrogation function time-to-time using the points traversed by the exploration process. The efficiency of derivative-free optimization thus depends on ways the two processes couple. In this paper, we propose a novel dynamical system, namely ThePrev---underline{S}tochastic underline{H}amiltonian underline{E}xploration and underline{E}xploitation, that surrogates the subregions of blackbox function using a time-evolving quadratic function, then explores and tracks the minimum of the quadratic functions using a fast-converging Hamiltonian system. The ThePrev algorithm is later provided as a discrete-time numerical approximation to the system. To further accelerate optimization, we present TheName that parallelizes multiple ThePrev threads for concurrent exploration and exploitation. Experiment results based on a wide range of machine learning applications show that TheName outperform a boarder range of derivative-free optimization algorithms with faster convergence speed under the same settings.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 Derivative-free optimization is not a novel domain and your work could benefit from some accepted benchmarking practices. For instance, you can consider the Black-Box Optimization Benchmarking (BBOB) Workshop and its COCO platform which was used to test many optimization algorithms including the ones mentioned in the paper. Benchmarking on BBOB problems would probably reveal that your algorithm fails on non-separable ill-conditioned problems and even on problems like Rosenbrock (e.g., compared to BOBYQA). The results for other algorithms can be downloaded, you don*t need to rerun them. BBOB*s computational budget can be as low as 1 function evaluation. Extended review of Update 17 Nov: I would like to note that I liked the fact that you used several optimization algorithms in your comparison. To my best understanding, several algorithms shown in Figure 3 (e.g., BOBYQA and L-BFGS) would benefit from restarts and it is fairly common to use restarts when the computational budget allows it (it seems to be the case for Figure 3). The results shown in Figure 4 are hard to trust because it does not seem that we observe mean/median results but probably a single run where the results after 1 iteration are drastically different for different algorithms. For instance, after one iteration BOBYQA only tests its second DOE point. Here, again, the issue is that one iteration for BOBYQA is 1 function evaluation while it is several (10) function evaluations for other algorithms. In that scenario, it would be more fair to run BOBYQA with 10 different initializations as well. I don*t understand "Due to the restriction of PyBOBYQA API, we can only provide the function evaluation of the final solution obtained by BOBYQA as a flatline in Figure 4". At the point when your objective function (which is not part of PyBOBYQA API) is called, I suppose you can log everything you need.
assistant
 Derivative-free optimization is not a novel domain and your work could benefit from some accepted benchmarking practices. For instance, you can consider the Black-Box Optimization Benchmarking (BBOB) Workshop and its COCO platform which was used to test many optimization algorithms including the ones mentioned in the paper. Benchmarking on BBOB problems would probably reveal that your algorithm fails on non-separable ill-conditioned problems and even on problems like Rosenbrock (e.g., compared to BOBYQA). The results for other algorithms can be downloaded, you don*t need to rerun them. BBOB*s computational budget can be as low as 1 function evaluation. Extended review of Update 17 Nov: I would like to note that I liked the fact that you used several optimization algorithms in your comparison. To my best understanding, several algorithms shown in Figure 3 (e.g., BOBYQA and L-BFGS) would benefit from restarts and it is fairly common to use restarts when the computational budget allows it (it seems to be the case for Figure 3). The results shown in Figure 4 are hard to trust because it does not seem that we observe mean/median results but probably a single run where the results after 1 iteration are drastically different for different algorithms. For instance, after one iteration BOBYQA only tests its second DOE point. Here, again, the issue is that one iteration for BOBYQA is 1 function evaluation while it is several (10) function evaluations for other algorithms. In that scenario, it would be more fair to run BOBYQA with 10 different initializations as well. I don*t understand "Due to the restriction of PyBOBYQA API, we can only provide the function evaluation of the final solution obtained by BOBYQA as a flatline in Figure 4". At the point when your objective function (which is not part of PyBOBYQA API) is called, I suppose you can log everything you need.