An Actor-critic Algorithm for Learning Rate Learning
Stochastic gradient descent (SGD), which updates the model parameters by adding a local gradient times a learning rate at each step, is widely used in model training of machine learning algorithms such as neural networks. It is observed that the models trained by SGD are sensitive to learning rates and good learning rates are problem specific. To avoid manually searching of learning rates, which is tedious and inefficient, we propose an algorithm to automatically learn learning rates using actor-critic methods from reinforcement learning (RL). In particular, we train a policy network called actor to decide the learning rate at each step during training, and a value network called critic to give feedback about quality of the decision (e.g., the goodness of the learning rate outputted by the actor) that the actor made. Experiments show that our method leads to good convergence of SGD and can prevent overfitting to a certain extent, resulting in better performance than human-designed competitors.
In the question response the authors mention and compare other works such as *Learning to Learn by Gradient Descent by Gradient Descent*, but the goal of current work and that work is quite different. That work is a new form of optimization algorithm which is not the case here. And bayesian hyper-parameter optimization methods aim for multiple hyper-parameters but this work only tune one hyper-parameter. The network architecture used for the experiments on CIFAR-10 is quite outdated and the performances are much poorer than any work that has published in last few years. So the comparison are not valid here, as if the paper claim the advantage of their method, they should use the state of the art network architecture and see if their claim still holds in that setting too. As discussed before, the extra cost of hyper-parameter optimizers are only justified if the method could push the SOTA results in multiple modern datasets. In summary, the general idea of having an actor-critic network as a meta-learner is an interesting idea. But the particular application proposed here does not seems to have any practical value and the reported results are very limited and it*s hard to draw any conclusion about the effectiveness of the method.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
An Actor-critic Algorithm for Learning Rate Learning
Stochastic gradient descent (SGD), which updates the model parameters by adding a local gradient times a learning rate at each step, is widely used in model training of machine learning algorithms such as neural networks. It is observed that the models trained by SGD are sensitive to learning rates and good learning rates are problem specific. To avoid manually searching of learning rates, which is tedious and inefficient, we propose an algorithm to automatically learn learning rates using actor-critic methods from reinforcement learning (RL). In particular, we train a policy network called actor to decide the learning rate at each step during training, and a value network called critic to give feedback about quality of the decision (e.g., the goodness of the learning rate outputted by the actor) that the actor made. Experiments show that our method leads to good convergence of SGD and can prevent overfitting to a certain extent, resulting in better performance than human-designed competitors.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



In the question response the authors mention and compare other works such as *Learning to Learn by Gradient Descent by Gradient Descent*, but the goal of current work and that work is quite different. That work is a new form of optimization algorithm which is not the case here. And bayesian hyper-parameter optimization methods aim for multiple hyper-parameters but this work only tune one hyper-parameter. The network architecture used for the experiments on CIFAR-10 is quite outdated and the performances are much poorer than any work that has published in last few years. So the comparison are not valid here, as if the paper claim the advantage of their method, they should use the state of the art network architecture and see if their claim still holds in that setting too. As discussed before, the extra cost of hyper-parameter optimizers are only justified if the method could push the SOTA results in multiple modern datasets. In summary, the general idea of having an actor-critic network as a meta-learner is an interesting idea. But the particular application proposed here does not seems to have any practical value and the reported results are very limited and it*s hard to draw any conclusion about the effectiveness of the method.
assistant
In the question response the authors mention and compare other works such as *Learning to Learn by Gradient Descent by Gradient Descent*, but the goal of current work and that work is quite different. That work is a new form of optimization algorithm which is not the case here. And bayesian hyper-parameter optimization methods aim for multiple hyper-parameters but this work only tune one hyper-parameter. The network architecture used for the experiments on CIFAR-10 is quite outdated and the performances are much poorer than any work that has published in last few years. So the comparison are not valid here, as if the paper claim the advantage of their method, they should use the state of the art network architecture and see if their claim still holds in that setting too. As discussed before, the extra cost of hyper-parameter optimizers are only justified if the method could push the SOTA results in multiple modern datasets. In summary, the general idea of having an actor-critic network as a meta-learner is an interesting idea. But the particular application proposed here does not seems to have any practical value and the reported results are very limited and it*s hard to draw any conclusion about the effectiveness of the method.