<p>Machine learning has been used to automatically learn new optimization/learning algorithms. This task is often known as <strong>meta-learning</strong>, i.e. you learn to learn, in this case, an optimization algorithm, but note that meta-learning does not just refer to learning optimization algorithms (see <a href="https://bair.berkeley.edu/blog/2017/07/18/learning-to-learn/" rel="nofollow noreferrer">this blog post</a>).</p>
<p>The blog post <a href="https://bair.berkeley.edu/blog/2017/09/12/learning-to-optimize-with-rl/" rel="nofollow noreferrer">Learning to Optimize with Reinforcement Learning</a> (2017) is a good introduction to the topic and focuses on the approach proposed in this paper <a href="https://arxiv.org/pdf/1606.01885.pdf" rel="nofollow noreferrer">Learning to Optimize</a> (2016), which uses reinforcement learning to solve this problem: more specifically, they learn a policy (in practice, represented as a neural network) that represents the learned optimization algorithm.</p>
<p>There are other related approaches: for example, you may be interested in the paper <a href="https://proceedings.neurips.cc/paper/2016/file/fb87582825f9d28a8d42c5e5e5e8b23d-Paper.pdf" rel="nofollow noreferrer">Learning to learn by gradient descent by gradient descent</a> (2016, NeurIPS).</p>

CC BY-SA 4.0
2021-11-29T17:11:39.053Z
2021-11-29T17:11:39.053Z
nbro
<p>Let's say that I want to create an optimization algorithm, which is supposed to find an optimum value for a given objective function. Creating an optimization algorithm to explore through the search space can be quite challenging.</p>
<p>My question is: can machine learning be used to automatically create optimization algorithms? Is there any source to look at for this?</p>

CC BY-SA 4.0
2021-11-29T16:09:24.447Z
if you can explain the challenges that you know, or even break it down into parts, it will give useful hint of some sort of structure for the answer
Sanyou
<machine-learning><optimization><algorithm-request><optimizers>
How do I use machine learning to create an optimization algorithm?
sherl.lol
# How do I use machine learning to create an optimization algorithm?

(Asked by: username_0 on 2021-11-29)

Let's say that I want to create an optimization algorithm, which is supposed to find an optimum value for a given objective function. Creating an optimization algorithm to explore through the search space can be quite challenging.

My question is: can machine learning be used to automatically create optimization algorithms? Is there any source to look at for this?

**Tags:** machine-learning, optimization, algorithm-request, optimizers

> **Comments:**
> **username_1**: if you can explain the challenges that you know, or even break it down into parts, it will give useful hint of some sort of structure for the answer  

---

## 1 answers

### Answer by username_2 on 2021-11-29. Score: 3 | Accepted answer

Machine learning has been used to automatically learn new optimization/learning algorithms. This task is often known as **meta-learning**, i.e. you learn to learn, in this case, an optimization algorithm, but note that meta-learning does not just refer to learning optimization algorithms (see [this blog post](https://bair.berkeley.edu/blog/2017/07/18/learning-to-learn/)).

The blog post [Learning to Optimize with Reinforcement Learning](https://bair.berkeley.edu/blog/2017/09/12/learning-to-optimize-with-rl/) (2017) is a good introduction to the topic and focuses on the approach proposed in this paper [Learning to Optimize](https://arxiv.org/pdf/1606.01885.pdf) (2016), which uses reinforcement learning to solve this problem: more specifically, they learn a policy (in practice, represented as a neural network) that represents the learned optimization algorithm.

There are other related approaches: for example, you may be interested in the paper [Learning to learn by gradient descent by gradient descent](https://proceedings.neurips.cc/paper/2016/file/fb87582825f9d28a8d42c5e5e5e8b23d-Paper.pdf) (2016, NeurIPS).


---

