Answers.Body: <p>NEAT is a <a href="https://en.wikipedia.org/wiki/Genetic_algorithm" rel="nofollow noreferrer">genetic algorithm</a> (GA). A genetic algorithm maintains a population of individuals (or chromosomes) and evolves it using operations like <a href="https://ai.stackexchange.com/a/9079/2444">the crossover or the mutation</a>, so that the fittest individuals keep living and most other individuals die. The nature of the individuals depends on the problem. For example, in the case of NEAT, the individuals are neural networks. However, these individuals need first to be encoded into a (compressed) reprentation (for example, a vector) that allows the operations like mutation to be efficiently applied: this representation is often called the <a href="https://en.wikipedia.org/wiki/Genotype%E2%80%93phenotype_distinction" rel="nofollow noreferrer">genotype</a> (or chromosome).</p>
  
  <p>How do you decide which individuals are the <a href="https://en.wikipedia.org/wiki/Survival_of_the_fittest" rel="nofollow noreferrer">fittest</a>? A function called the <a href="https://en.wikipedia.org/wiki/Fitness_function" rel="nofollow noreferrer">fitness function</a> needs first to be defined. The fitness function measures the fitness (or quality) of the individuals (or solutions). In the case of neural networks, a fitness function could, for example, be the accuracy of the neural networks on a validation dataset. </p>
  
  <p>Why don't GAs need gradients? The fitness (or quality) of the solutions is given by the fitness function in the case of GAs, so gradients are not strictly needed, even though they can also be used.</p>
  
  <p>NEAT is a little bit more complex (and it is described in detail in the paper that introduced it <a href="http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf" rel="nofollow noreferrer">Evolving Neural Networks through Augmenting Topologies</a>), but this is the basic idea behind all genetic algorithms (including NEAT).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-04T23:13:45.340Z
Answers.LastActivityDate: 2019-06-05T21:45:32.850Z
Answers.LastEditDate: 2019-06-05T21:45:32.850Z
Answers.UserDisplayName: nbro
Body: <p>I'm new to NEAT, so, please, don't be too harsh. How does NEAT find the most successful generation without gradient descent or gradients?</p>
  
Tags: <genetic-algorithms><optimization><gradient-descent><neat>
Title: How does NEAT find the most successful generation without gradients?
UserDisplayName: Sebastian Dixon
ThreadText: # How does NEAT find the most successful generation without gradients?
  
  (Asked by: username_0 on 2019-06-04)
  
  I'm new to NEAT, so, please, don't be too harsh. How does NEAT find the most successful generation without gradient descent or gradients?
  
  **Tags:** genetic-algorithms, optimization, gradient-descent, neat
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-06-04. Score: 3 | Accepted answer
  
  NEAT is a [genetic algorithm](https://en.wikipedia.org/wiki/Genetic_algorithm) (GA). A genetic algorithm maintains a population of individuals (or chromosomes) and evolves it using operations like [the crossover or the mutation](https://ai.stackexchange.com/a/9079/2444), so that the fittest individuals keep living and most other individuals die. The nature of the individuals depends on the problem. For example, in the case of NEAT, the individuals are neural networks. However, these individuals need first to be encoded into a (compressed) reprentation (for example, a vector) that allows the operations like mutation to be efficiently applied: this representation is often called the [genotype](https://en.wikipedia.org/wiki/Genotype%E2%80%93phenotype_distinction) (or chromosome).
  
  How do you decide which individuals are the [fittest](https://en.wikipedia.org/wiki/Survival_of_the_fittest)? A function called the [fitness function](https://en.wikipedia.org/wiki/Fitness_function) needs first to be defined. The fitness function measures the fitness (or quality) of the individuals (or solutions). In the case of neural networks, a fitness function could, for example, be the accuracy of the neural networks on a validation dataset.
  
  Why don't GAs need gradients? The fitness (or quality) of the solutions is given by the fitness function in the case of GAs, so gradients are not strictly needed, even though they can also be used.
  
  NEAT is a little bit more complex (and it is described in detail in the paper that introduced it [Evolving Neural Networks through Augmenting Topologies](http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf)), but this is the basic idea behind all genetic algorithms (including NEAT).
  
  
  ---
  
  