Answers.Body: <blockquote>
    <p>Is such an idea viable? </p>
  </blockquote>
  
  <p>Yes. </p>
  
  <p>One approach that <em>should</em> work in terms of underlying theory is to start with population of NEAT-generated networks that describe the policy, and instead of measuring fitness of them on the task whilst keeping all weights static, measure fitness whilst applying any policy gradient algorithm like A3C. In addition, the final weights of the networks could be fed into the next generation. <a href="https://ieeexplore.ieee.org/document/4637766/" rel="nofollow noreferrer">A bit Lamarckian* perhaps, but that is already a thing in evolutionary algorithms</a>.</p>
  
  <blockquote>
    <p>Has it been tried? </p>
  </blockquote>
  
  <p>Yes, <a href="https://dl.acm.org/citation.cfm?id=3205455.3205536" rel="nofollow noreferrer">a recent paper (July 2018) is "NEAT for large-scale reinforcement learning through evolutionary feature learning and policy gradient search"</a>. </p>
  
  <p>I suspect there are more, including hobby efforts without academic publishing, but that is the first paper that popped up on a brief search.</p>
  
  <hr>
  
  <p>* <a href="https://en.wikipedia.org/wiki/Jean-Baptiste_Lamarck" rel="nofollow noreferrer">Jean-Baptiste Lamarck</a> believed that offspring of animal parents could inherit traits based on parental behaviour and desires, such as a giraffe's neck evolving over time as the animals strived to reach higher food sources. Interestingly, although the core of this theory is not generally accepted, recent theories around phenotypic plasticity and discoveries in epigenetics show that biological systems can make use of the idea - although very far from the idea of children using parental memories directly to aid in tasks (which has more in common with science fiction for humans, but possible in neural networks where we can "copy brains").</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-29T16:58:09.540Z
Answers.LastActivityDate: 2018-07-29T17:06:10.573Z
Answers.LastEditDate: 2018-07-29T17:06:10.573Z
Answers.UserDisplayName: Neil Slater
Body: <p>As a amateur researcher and tinkerer, I've been reading up on neuro-evolution networks (e.g. NEAT) as well as the A3C RL approach presented by <a href="https://arxiv.org/pdf/1602.01783v1.pdf" rel="nofollow noreferrer">Mnih et al</a> and got to wondering if anyone has contemplated the merging of both these techniques.</p>
  
  <p>Is such an idea viable? Has it been tried? </p>
  
  <p>I'd be interested in any research in this area as it sounds like it could be compelling.</p>
  
Tags: <reinforcement-learning><genetic-algorithms><neat><actor-critic-methods>
Title: Can neuro-evolution methods be combined with A3C?
UserDisplayName: Brad
ThreadText: # Can neuro-evolution methods be combined with A3C?
  
  (Asked by: username_0 on 2018-07-29)
  
  As a amateur researcher and tinkerer, I've been reading up on neuro-evolution networks (e.g. NEAT) as well as the A3C RL approach presented by [Mnih et al](https://arxiv.org/pdf/1602.01783v1.pdf) and got to wondering if anyone has contemplated the merging of both these techniques.
  
  Is such an idea viable? Has it been tried?
  
  I'd be interested in any research in this area as it sounds like it could be compelling.
  
  **Tags:** reinforcement-learning, genetic-algorithms, neat, actor-critic-methods
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-07-29. Score: 1
  
  > Is such an idea viable?
  
  Yes.
  
  One approach that *should* work in terms of underlying theory is to start with population of NEAT-generated networks that describe the policy, and instead of measuring fitness of them on the task whilst keeping all weights static, measure fitness whilst applying any policy gradient algorithm like A3C. In addition, the final weights of the networks could be fed into the next generation. [A bit Lamarckian\* perhaps, but that is already a thing in evolutionary algorithms](https://ieeexplore.ieee.org/document/4637766/).
  
  > Has it been tried?
  
  Yes, [a recent paper (July 2018) is "NEAT for large-scale reinforcement learning through evolutionary feature learning and policy gradient search"](https://dl.acm.org/citation.cfm?id=3205455.3205536).
  
  I suspect there are more, including hobby efforts without academic publishing, but that is the first paper that popped up on a brief search.
  
  ---
  
  \* [Jean-Baptiste Lamarck](https://en.wikipedia.org/wiki/Jean-Baptiste_Lamarck) believed that offspring of animal parents could inherit traits based on parental behaviour and desires, such as a giraffe's neck evolving over time as the animals strived to reach higher food sources. Interestingly, although the core of this theory is not generally accepted, recent theories around phenotypic plasticity and discoveries in epigenetics show that biological systems can make use of the idea - although very far from the idea of children using parental memories directly to aid in tasks (which has more in common with science fiction for humans, but possible in neural networks where we can "copy brains").
  
  
  ---
  
  