Answers.Body: <p>There are a few advantages of genetic algorithms (GAs) over neural networks using gradient descent for solving optimisation problems:</p>
  <ul>
  <li><p>GAs don't require relationship between any objective function (fitness function) and learnable parameters to be differentiable.</p>
  </li>
  <li><p>GAs can alter hyperparameters of a model structure. This is related to the first point, but worth mentioning because it allows a GA like <a href="https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies" rel="nofollow noreferrer">the NEAT algorithm</a> to alter structure of neural networks on the fly.</p>
  </li>
  <li><p>GAs can be used to optimise models where the relationship between the models' outputs and end results are unclear, but those end results can be assessed. This is a property shared with reinforcement learning.</p>
  </li>
  </ul>
  <p>Unfortunately, this extra flexibility comes at a cost. Compared to gradient-based algorithms, GAs are often inefficient and slow. When a problem requires a model with a large number of free parameters, then GAs become impractical.</p>
  <p>Regarding a requirement for lots of data and compute time;</p>
  <blockquote>
  <p>Can genetic programming relieve me of these hassles?</p>
  </blockquote>
  <p>Not usually. In some cases it might happen that a small non-differential model neatly encapsulates the problem and a GA or GP could find it. But for most hard problems where large neural networks have been applied, there are no known alternatives than gradient-based learning algorithms that require significant computation and lots of data.</p>
  <p>You may still have alternatives for specific problems in practice. However researching a hard data-intense problem from scratch using a GA could well require orders of magnitude more compute resources than gradient-based methods.</p>
  <blockquote>
  <p>I am particularly interested in the protein structure prediction problem.</p>
  </blockquote>
  <p>There might be a role for GA or GP algorithms in searching for solutions to this problem. However, trying to frame the entire search as GA or GP is not practical. Instead a GA might be useful to <em>direct</em> model creation and testing.</p>
  <p>There's no avoiding the significant difficulties and compute costs for research into the issue. At least not without a theoretical breakthrough in the core science of understanding how protein structures are created at the atomic level.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-08T08:56:09.757Z
Answers.LastActivityDate: 2024-02-08T09:14:26.790Z
Answers.LastEditDate: 2024-02-08T09:14:26.790Z
Answers.UserDisplayName: Neil Slater
Body: <p>I am planning to switch from neural networks to genetic algorithms (GA) and programming (GP).</p>
  <p>One of the main hassles of working with neural networks is that it requires a large amount of training data. Another issue with neural networks is that they require powerful computers for training, which I cannot afford.</p>
  <p>Can genetic programming relieve me of these hassles?</p>
  <p>I'm particularly interested in the <em>protein structure prediction</em> problem.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-02T04:59:12.847Z
Comments.Text: Your question is too broad. Please provide some clarity on the problem you want to solve. Explain your rationale for considering a GA or GP. Solutions based on GAs and GPs can be very time-consuming and may require a cluster of computers that are far more expensive than a retail GPU if you want to finish the problem quickly.
Comments.UserDisplayName: Brian O'Donnell
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-02T08:02:33.943Z
Comments.Text: Could you give an example of a problem that you want to solve? Preferably one where you have rejected neural networks as too slow and data-hungry.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-02T08:03:08.347Z
Comments.Text: @NeilSlater, Protein structure prediction.
Comments.UserDisplayName: user366312
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-02T09:44:51.963Z
Comments.Text: They are doing different things. NNs are usually used for classification tasks, whereas GAs are for optimising a configuration etc.
Comments.UserDisplayName: Oliver Mason
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-03T01:55:06.630Z
Comments.Text: Did you research the use of GA or GP for protein structure prediction? If so, what did you learn? A good paper should compare its approach to some baseline or alternative approaches, which will likely include neural networks.
Comments.UserDisplayName: Brian O'Donnell
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-06T12:28:11.613Z
Comments.Text: @BrianO'Donnell Just vote to close the post as "details details or clarity" or "needs more focus". You've been on this site for so long, you should already know this.
Comments.UserDisplayName: nbro
Tags: <neural-networks><comparison><genetic-algorithms><genetic-programming>
Title: Is there any advantage of genetic algorithm (or programming) over Neural Networks?
UserDisplayName: user366312
ThreadText: # Is there any advantage of genetic algorithm (or programming) over Neural Networks?
  
  (Asked by: username_0 on 2024-02-02)
  
  I am planning to switch from neural networks to genetic algorithms (GA) and programming (GP).
  
  One of the main hassles of working with neural networks is that it requires a large amount of training data. Another issue with neural networks is that they require powerful computers for training, which I cannot afford.
  
  Can genetic programming relieve me of these hassles?
  
  I'm particularly interested in the *protein structure prediction* problem.
  
  **Tags:** neural-networks, comparison, genetic-algorithms, genetic-programming
  
  > **Comments:**
  > **username_1**: Your question is too broad. Please provide some clarity on the problem you want to solve. Explain your rationale for considering a GA or GP. Solutions based on GAs and GPs can be very time-consuming and may require a cluster of computers that are far more expensive than a retail GPU if you want to finish the problem quickly.  
  > **username_2**: Could you give an example of a problem that you want to solve? Preferably one where you have rejected neural networks as too slow and data-hungry.  
  > **username_0**: @username_2, Protein structure prediction.  
  > **username_3**: They are doing different things. NNs are usually used for classification tasks, whereas GAs are for optimising a configuration etc.  
  > **username_1**: Did you research the use of GA or GP for protein structure prediction? If so, what did you learn? A good paper should compare its approach to some baseline or alternative approaches, which will likely include neural networks.  
  > **username_4**: @username_1 Just vote to close the post as "details details or clarity" or "needs more focus". You've been on this site for so long, you should already know this.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2024-02-08. Score: 1 | Accepted answer
  
  There are a few advantages of genetic algorithms (GAs) over neural networks using gradient descent for solving optimisation problems:
  
  * GAs don't require relationship between any objective function (fitness function) and learnable parameters to be differentiable.
  * GAs can alter hyperparameters of a model structure. This is related to the first point, but worth mentioning because it allows a GA like [the NEAT algorithm](https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies) to alter structure of neural networks on the fly.
  * GAs can be used to optimise models where the relationship between the models' outputs and end results are unclear, but those end results can be assessed. This is a property shared with reinforcement learning.
  
  Unfortunately, this extra flexibility comes at a cost. Compared to gradient-based algorithms, GAs are often inefficient and slow. When a problem requires a model with a large number of free parameters, then GAs become impractical.
  
  Regarding a requirement for lots of data and compute time;
  
  > Can genetic programming relieve me of these hassles?
  
  Not usually. In some cases it might happen that a small non-differential model neatly encapsulates the problem and a GA or GP could find it. But for most hard problems where large neural networks have been applied, there are no known alternatives than gradient-based learning algorithms that require significant computation and lots of data.
  
  You may still have alternatives for specific problems in practice. However researching a hard data-intense problem from scratch using a GA could well require orders of magnitude more compute resources than gradient-based methods.
  
  > I am particularly interested in the protein structure prediction problem.
  
  There might be a role for GA or GP algorithms in searching for solutions to this problem. However, trying to frame the entire search as GA or GP is not practical. Instead a GA might be useful to *direct* model creation and testing.
  
  There's no avoiding the significant difficulties and compute costs for research into the issue. At least not without a theoretical breakthrough in the core science of understanding how protein structures are created at the atomic level.
  
  
  ---
  
  