Answers.Body: <p>OpenAI's ES updates parameters using learning rate. It may remain constant, decaying or changed in different ways. You can for example use momentum that's commonly used in stochastic gradient descent. So Adam is used to adjust learning rates per parameters.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-12T17:38:12.820Z
Answers.Text: Oh that's interesting. Just checking though you're sure about this?
  Could you add some citations?
Answers.UserDisplayName: profPlum
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-12T18:53:41.077Z
Answers.Text: Not citations per se, but look at the code in [github](https://github.com/hardmaru/estool/blob/master/es.py#L331) from [Visual Guide to Evolution Strategies](https://blog.otoro.net/2017/10/29/visual-evolution-strategies/).
Answers.UserDisplayName: derjack
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-12T17:13:44.687Z
Answers.LastActivityDate: 2022-05-12T17:13:44.687Z
Answers.UserDisplayName: derjack
Body: <p>I just read that OpenAI's ES uses Adam: <a href="https://www.goodai.com/distributed-evolutionary-computation-on-rl-tasks/" rel="nofollow noreferrer">&quot;OpenAI’s ES is denoted as “OptimES” (since it uses Adam optimizer)&quot;</a>?? I verified <a href="https://github.com/openai/evolution-strategies-starter" rel="nofollow noreferrer">they are correct</a> using the link they posted, (see es_distributed/Optimizers.py). But I don't understand how because the paper says they are using <a href="https://arxiv.org/pdf/1703.03864.pdf" rel="nofollow noreferrer">Evolution Strategies as a Scalable Alternative to Reinforcement Learning</a>, which is black box optimization (like most ES's)... So how on earth is Adam used, given this is black box?</p>
  
Tags: <reinforcement-learning><evolutionary-algorithms><neuroevolution><adam>
Title: How does OpenAI-ES use Adam?
UserDisplayName: profPlum
ThreadText: # How does OpenAI-ES use Adam?
  
  (Asked by: username_0 on 2022-03-24)
  
  I just read that OpenAI's ES uses Adam: ["OpenAI’s ES is denoted as “OptimES” (since it uses Adam optimizer)"](https://www.goodai.com/distributed-evolutionary-computation-on-rl-tasks/)?? I verified [they are correct](https://github.com/openai/evolution-strategies-starter) using the link they posted, (see es\_distributed/Optimizers.py). But I don't understand how because the paper says they are using [Evolution Strategies as a Scalable Alternative to Reinforcement Learning](https://arxiv.org/pdf/1703.03864.pdf), which is black box optimization (like most ES's)... So how on earth is Adam used, given this is black box?
  
  **Tags:** reinforcement-learning, evolutionary-algorithms, neuroevolution, adam
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-05-12. Score: 0
  
  OpenAI's ES updates parameters using learning rate. It may remain constant, decaying or changed in different ways. You can for example use momentum that's commonly used in stochastic gradient descent. So Adam is used to adjust learning rates per parameters.
  
  > **Comments:**
  > **username_0**: Oh that's interesting. Just checking though you're sure about this? Could you add some citations?  
  > **username_1**: Not citations per se, but look at the code in [github](https://github.com/hardmaru/estool/blob/master/es.py#L331) from [Visual Guide to Evolution Strategies](https://blog.otoro.net/2017/10/29/visual-evolution-strategies/).  
  
  ---
  
  