Answers.Body: <p>NEAT uses genetic algorithms both to search for improved connection weights and for improved architectures.</p>
  
  <p>Whilst it is <em>possible</em> to train a NEAT-generated neural network using backpropagation of error gradients, libraries implementing "original" NEAT will not implement that.</p>
  
  <p>There are a couple of reasons:</p>
  
  <ul>
  <li><p>There is often no training data, in a supervised learning sense. The fitness function of a NEAT system can be measured arbitrarily by performance at some task, and in the general case this consists of some <em>environment</em> simulation that interacts with an agent controlled by the NN, and not training data per se.</p></li>
  <li><p>Evolved network topologies do not conform to stacked layers models preferred by ML frameworks designed to run typical deep learning architectures.</p></li>
  </ul>
  
  <p>Both of these issues can be resolved with a little effort (for instance for the second issue there are frameworks which will work with arbitrary feed forward connection graphs, they are just a little more niche than e.g. Keras). However, NEAT is often used because it can solve problems without needing to frame them as supervised learning or reinforcement learning.</p>
  
  <p>Other than the hard work of putting it together, there is nothing stopping you creating a train-on-data stage which could alternate between the two approaches, perhaps with controllable weighting from evolution to gradient-based training. To add the gradient-based training, then either:</p>
  
  <p>a) Your original problem is one of fitting to a classifier or regression problem. In which case your fitness function and training loss function could be the same.</p>
  
  <p>b) Your original problem is one of controlling an agent in an environment, and you could potentially use something like the REINFORCE algorithm based on most-recent assessments, in order to provide gradients to train the NN. Other policy gradient methods could also be a good fit, as the NEAT network typically outputs a policy as opposed to a value prediction.</p>
  
  <p>I have never tried these (nor much with NEAT at all other than demos and a bit of theory). For (a) I would expect the combination to be successful, but wonder why you were bothering with NEAT in the first place. For (b) I am less sure whether you would get useful results, because REINFORCE relies on multiple runs with the <em>same</em> network, whilst NEAT relies on stochastic search across multiple networks. Applying REINFORCE training across a whole population could be very CPU intensive.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-03-09T19:38:08.070Z
Answers.LastActivityDate: 2018-03-09T19:53:14.837Z
Answers.LastEditDate: 2018-03-09T19:53:14.837Z
Answers.UserDisplayName: Neil Slater
Body: <p>When one uses NEAT to evolve the best fitting network for a task, does training take place in each epoch as well?</p>
  
  <p>If I understand correctly, training is the adjustment of the weights of the neural network via backpropagation and gradient descent. During NEAT, say, a generation runs for 1000 iterations. During that time, is there any training involved, or does each genome randomly poke around and the winner takes it to the next stage?</p>
  
  <p>I've used NEAT, but the fact that neural networks are not trained does not make sense to me. At the same time, I can't find any code in my framework (<a href="https://wagenaartje.github.io/neataptic/" rel="nofollow noreferrer">Neataptic.js</a>) that would train the generation during the epoch.</p>
  
Tags: <neural-networks><training><backpropagation><neat><neuroevolution>
Title: Does training happen during NEAT?
UserDisplayName: Alexus
ThreadText: # Does training happen during NEAT?
  
  (Asked by: username_0 on 2018-03-08)
  
  When one uses NEAT to evolve the best fitting network for a task, does training take place in each epoch as well?
  
  If I understand correctly, training is the adjustment of the weights of the neural network via backpropagation and gradient descent. During NEAT, say, a generation runs for 1000 iterations. During that time, is there any training involved, or does each genome randomly poke around and the winner takes it to the next stage?
  
  I've used NEAT, but the fact that neural networks are not trained does not make sense to me. At the same time, I can't find any code in my framework ([Neataptic.js](https://wagenaartje.github.io/neataptic/)) that would train the generation during the epoch.
  
  **Tags:** neural-networks, training, backpropagation, neat, neuroevolution
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-03-09. Score: 4
  
  NEAT uses genetic algorithms both to search for improved connection weights and for improved architectures.
  
  Whilst it is *possible* to train a NEAT-generated neural network using backpropagation of error gradients, libraries implementing "original" NEAT will not implement that.
  
  There are a couple of reasons:
  
  * There is often no training data, in a supervised learning sense. The fitness function of a NEAT system can be measured arbitrarily by performance at some task, and in the general case this consists of some *environment* simulation that interacts with an agent controlled by the NN, and not training data per se.
  * Evolved network topologies do not conform to stacked layers models preferred by ML frameworks designed to run typical deep learning architectures.
  
  Both of these issues can be resolved with a little effort (for instance for the second issue there are frameworks which will work with arbitrary feed forward connection graphs, they are just a little more niche than e.g. Keras). However, NEAT is often used because it can solve problems without needing to frame them as supervised learning or reinforcement learning.
  
  Other than the hard work of putting it together, there is nothing stopping you creating a train-on-data stage which could alternate between the two approaches, perhaps with controllable weighting from evolution to gradient-based training. To add the gradient-based training, then either:
  
  a) Your original problem is one of fitting to a classifier or regression problem. In which case your fitness function and training loss function could be the same.
  
  b) Your original problem is one of controlling an agent in an environment, and you could potentially use something like the REINFORCE algorithm based on most-recent assessments, in order to provide gradients to train the NN. Other policy gradient methods could also be a good fit, as the NEAT network typically outputs a policy as opposed to a value prediction.
  
  I have never tried these (nor much with NEAT at all other than demos and a bit of theory). For (a) I would expect the combination to be successful, but wonder why you were bothering with NEAT in the first place. For (b) I am less sure whether you would get useful results, because REINFORCE relies on multiple runs with the *same* network, whilst NEAT relies on stochastic search across multiple networks. Applying REINFORCE training across a whole population could be very CPU intensive.
  
  
  ---
  
  