Answers.Body: <p>Generally speaking, when architecture search is essential or the problem objective is non-differentiable while creativity and diversity in solutions matter most, use NEAT/GA. When rewards are well-defined, environments are complex but problem objective is differentiable, and ML model topology is fixed, use RL.</p>
  <p>Having said that, there're RL approaches changing the topology of neural networks during training. One popular framework is <a href="https://en.wikipedia.org/wiki/Neural_architecture_search" rel="nofollow noreferrer">NAS/AutoML</a> which employs RL to adapt and optimize ANN topology.</p>
  <blockquote>
  <p>Reinforcement learning (RL) can underpin a NAS search strategy. Barret Zoph and Quoc Viet Le applied NAS with RL targeting the CIFAR-10 dataset and achieved a network architecture that rivals the best manually-designed architecture for accuracy... In the so-called Efficient Neural Architecture Search (ENAS), a controller discovers architectures by learning to search for an optimal subgraph within a large graph. The controller is trained with policy gradient to select a subgraph that maximizes the validation set's expected reward.</p>
  </blockquote>
  <p>In some cases NAS variant also employs evolutionary algorithms including GA to adapt and optimize ANN topology.</p>
  <blockquote>
  <p>An Evolutionary Algorithm for Neural Architecture Search generally performs the following procedure... On CIFAR-10 and ImageNet, evolution and RL performed comparably, while both slightly outperformed random search.</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-05T10:46:32.143Z
Answers.Text: To be sure, by GA we mean crossing and changing the weights of neural networks while leaving the topology unchanged?
Answers.UserDisplayName: Nikolai Vorobiev
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-05T18:41:14.597Z
Answers.Text: Not necessarily, here meant for your concerned hyperparameters optimization of topology of ANN, see [GA](https://en.wikipedia.org/wiki/Genetic_algorithm) definition: *Genetic algorithms are commonly used to generate high-quality solutions to optimization and search problems via biologically inspired operators such as selection, crossover, and mutation. Some examples of GA applications include optimizing decision trees for better performance, solving sudoku puzzles, hyperparameter optimization, and causal inference.* Hope this clarifies your lingering concern and helpful for your question.
Answers.UserDisplayName: cinch
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-05T20:07:01.643Z
Answers.Text: Thank you, I understand that GA is a more general concept that includes NEAT?  And I can't remember when a problem is not differentiable for NN, give a couple of examples
Answers.UserDisplayName: Nikolai Vorobiev
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-05T21:28:58.210Z
Answers.Text: If you only restricted to NN with non-differentiable objective, my above answer deals with hyperparameter optimization of ANN topologies for your OP since number of layers, types of activations, adding/removing nodes are all non-differentiable. Also even in RL such as AlphaZero, you still need MCTS heuristics where there's no gradient to backprop. And your loss in some cases could be F1 or BLEU (for language generation) liek non-differentiable functions. Hope this clarifies your lingering question, if you have more related questions, you could post new question.
Answers.UserDisplayName: cinch
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-07T07:48:45.467Z
Answers.Text: so RL uses gradient descent and for this the task should be differentiable, however, as You mentioned we have AutoML, so now I am confused - if RL requires differentiability, but also, as you said, there are solutions for non-differentiable questions with reinforcement learning, I am completely lost in the algorithm's operation
Answers.UserDisplayName: Nikolai Vorobiev
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-07T07:53:22.217Z
Answers.Text: Not surprised by your lost since RL is a wide area in AI and complex algos such as AlphaZero contains RL and MCTS and other tricks, but for simplicity sometimes we classify it as RL. But indeed you may have F1 or BLEU loss functions in some rare case, then such RL is not differentiable. So the claim that *RL requires differentiability* is incorrect strictly speaking. Hope this finally clarifies and useful to you.
Answers.UserDisplayName: cinch
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-05T07:52:25.303Z
Answers.LastActivityDate: 2024-12-05T07:52:25.303Z
Answers.UserDisplayName: cinch
Body: <p>When we should use NEAT and GA? Today the problem of “We do not know which answer is correct” was solved by Reinforcement Learning, so when using genetic algorithm is still useful, and do we have RL that changes topology of NN?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-12-05T11:01:33.653Z
Comments.Text: 'Today the problem of “We do not know which answer is correct” was solved by Reinforcement Learning' - What? That's a very unclear way of phrasing what RL does. Can you clarify what you mean by "why when using genetic algorithm is still useful"? Why or when? It's unclear. This post is so confusing and it's unclear why you're asking the question in the title, which is the only thing that is more or less clear.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-12-05T11:33:09.530Z
Comments.Text: That is why = so
Comments.UserDisplayName: Nikolai Vorobiev
Tags: <neural-networks><reinforcement-learning><genetic-algorithms><neat><education>
Title: When we should use NEAT and GA?
UserDisplayName: Nikolai Vorobiev
ThreadText: # When we should use NEAT and GA?
  
  (Asked by: username_0 on 2024-12-05)
  
  When we should use NEAT and GA? Today the problem of “We do not know which answer is correct” was solved by Reinforcement Learning, so when using genetic algorithm is still useful, and do we have RL that changes topology of NN?
  
  **Tags:** neural-networks, reinforcement-learning, genetic-algorithms, neat, education
  > **Comments:**
  > **username_1**: 'Today the problem of “We do not know which answer is correct” was solved by Reinforcement Learning' - What? That's a very unclear way of phrasing what RL does. Can you clarify what you mean by "why when using genetic algorithm is still useful"? Why or when? It's unclear. This post is so confusing and it's unclear why you're asking the question in the title, which is the only thing that is more or less clear.  
  > **username_0**: That is why = so  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2024-12-05. Score: 0 | Accepted answer
  
  Generally speaking, when architecture search is essential or the problem objective is non-differentiable while creativity and diversity in solutions matter most, use NEAT/GA. When rewards are well-defined, environments are complex but problem objective is differentiable, and ML model topology is fixed, use RL.
  
  Having said that, there're RL approaches changing the topology of neural networks during training. One popular framework is [NAS/AutoML](https://en.wikipedia.org/wiki/Neural_architecture_search) which employs RL to adapt and optimize ANN topology.
  
  > Reinforcement learning (RL) can underpin a NAS search strategy. Barret Zoph and Quoc Viet Le applied NAS with RL targeting the CIFAR-10 dataset and achieved a network architecture that rivals the best manually-designed architecture for accuracy... In the so-called Efficient Neural Architecture Search (ENAS), a controller discovers architectures by learning to search for an optimal subgraph within a large graph. The controller is trained with policy gradient to select a subgraph that maximizes the validation set's expected reward.
  
  In some cases NAS variant also employs evolutionary algorithms including GA to adapt and optimize ANN topology.
  
  > An Evolutionary Algorithm for Neural Architecture Search generally performs the following procedure... On CIFAR-10 and ImageNet, evolution and RL performed comparably, while both slightly outperformed random search.
  
  > **Comments:**
  > **username_0**: To be sure, by GA we mean crossing and changing the weights of neural networks while leaving the topology unchanged?  
  > **username_2**: Not necessarily, here meant for your concerned hyperparameters optimization of topology of ANN, see [GA](https://en.wikipedia.org/wiki/Genetic_algorithm) definition: *Genetic algorithms are commonly used to generate high-quality solutions to optimization and search problems via biologically inspired operators such as selection, crossover, and mutation. Some examples of GA applications include optimizing decision trees for better performance, solving sudoku puzzles, hyperparameter optimization, and causal inference.* Hope this clarifies your lingering concern and helpful for your question.  
  > **username_0**: Thank you, I understand that GA is a more general concept that includes NEAT?  And I can't remember when a problem is not differentiable for NN, give a couple of examples  
  > **username_2**: If you only restricted to NN with non-differentiable objective, my above answer deals with hyperparameter optimization of ANN topologies for your OP since number of layers, types of activations, adding/removing nodes are all non-differentiable. Also even in RL such as AlphaZero, you still need MCTS heuristics where there's no gradient to backprop. And your loss in some cases could be F1 or BLEU (for language generation) liek non-differentiable functions. Hope this clarifies your lingering question, if you have more related questions, you could post new question.  
  > **username_0**: so RL uses gradient descent and for this the task should be differentiable, however, as You mentioned we have AutoML, so now I am confused - if RL requires differentiability, but also, as you said, there are solutions for non-differentiable questions with reinforcement learning, I am completely lost in the algorithm's operation  
  > **username_2**: Not surprised by your lost since RL is a wide area in AI and complex algos such as AlphaZero contains RL and MCTS and other tricks, but for simplicity sometimes we classify it as RL. But indeed you may have F1 or BLEU loss functions in some rare case, then such RL is not differentiable. So the claim that *RL requires differentiability* is incorrect strictly speaking. Hope this finally clarifies and useful to you.  
  
  ---
  
  