Answers.Body: <p>Machine learning has been used to automatically learn new optimization/learning algorithms. This task is often known as <strong>meta-learning</strong>, i.e. you learn to learn, in this case, an optimization algorithm, but note that meta-learning does not just refer to learning optimization algorithms (see <a href="https://bair.berkeley.edu/blog/2017/07/18/learning-to-learn/" rel="nofollow noreferrer">this blog post</a>).</p>
  <p>The blog post <a href="https://bair.berkeley.edu/blog/2017/09/12/learning-to-optimize-with-rl/" rel="nofollow noreferrer">Learning to Optimize with Reinforcement Learning</a> (2017) is a good introduction to the topic and focuses on the approach proposed in this paper <a href="https://arxiv.org/pdf/1606.01885.pdf" rel="nofollow noreferrer">Learning to Optimize</a> (2016), which uses reinforcement learning to solve this problem: more specifically, they learn a policy (in practice, represented as a neural network) that represents the learned optimization algorithm.</p>
  <p>There are other related approaches: for example, you may be interested in the paper <a href="https://proceedings.neurips.cc/paper/2016/file/fb87582825f9d28a8d42c5e5e5e8b23d-Paper.pdf" rel="nofollow noreferrer">Learning to learn by gradient descent by gradient descent</a> (2016, NeurIPS).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-11-29T17:11:39.053Z
Answers.LastActivityDate: 2021-11-29T17:11:39.053Z
Answers.UserDisplayName: nbro
Body: <p>Let's say that I want to create an optimization algorithm, which is supposed to find an optimum value for a given objective function. Creating an optimization algorithm to explore through the search space can be quite challenging.</p>
  <p>My question is: can machine learning be used to automatically create optimization algorithms? Is there any source to look at for this?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-11-29T16:09:24.447Z
Comments.Text: if you can explain the challenges that you know, or even break it down into parts, it will give useful hint of some sort of structure for the answer
Comments.UserDisplayName: Sanyou
Tags: <machine-learning><optimization><algorithm-request><optimizers>
Title: How do I use machine learning to create an optimization algorithm?
UserDisplayName: sherl.lol
ThreadText: # How do I use machine learning to create an optimization algorithm?
  
  (Asked by: username_0 on 2021-11-29)
  
  Let's say that I want to create an optimization algorithm, which is supposed to find an optimum value for a given objective function. Creating an optimization algorithm to explore through the search space can be quite challenging.
  
  My question is: can machine learning be used to automatically create optimization algorithms? Is there any source to look at for this?
  
  **Tags:** machine-learning, optimization, algorithm-request, optimizers
  > **Comments:**
  > **username_1**: if you can explain the challenges that you know, or even break it down into parts, it will give useful hint of some sort of structure for the answer  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-11-29. Score: 3 | Accepted answer
  
  Machine learning has been used to automatically learn new optimization/learning algorithms. This task is often known as **meta-learning**, i.e. you learn to learn, in this case, an optimization algorithm, but note that meta-learning does not just refer to learning optimization algorithms (see [this blog post](https://bair.berkeley.edu/blog/2017/07/18/learning-to-learn/)).
  
  The blog post [Learning to Optimize with Reinforcement Learning](https://bair.berkeley.edu/blog/2017/09/12/learning-to-optimize-with-rl/) (2017) is a good introduction to the topic and focuses on the approach proposed in this paper [Learning to Optimize](https://arxiv.org/pdf/1606.01885.pdf) (2016), which uses reinforcement learning to solve this problem: more specifically, they learn a policy (in practice, represented as a neural network) that represents the learned optimization algorithm.
  
  There are other related approaches: for example, you may be interested in the paper [Learning to learn by gradient descent by gradient descent](https://proceedings.neurips.cc/paper/2016/file/fb87582825f9d28a8d42c5e5e5e8b23d-Paper.pdf) (2016, NeurIPS).
  
  
  ---
  
  