Answers.Body: <p><a href="https://ai.stackexchange.com/a/1399/2444">The other answer</a> mentions <a href="https://www.cs.ucf.edu/%7Ekstanley/neat.html" rel="nofollow noreferrer">NEAT</a> to generate network weights or topologies. The paper <a href="http://doc.gold.ac.uk/aisb50/AISB50-S11/AISB50-S11-Turner-paper.pdf" rel="nofollow noreferrer">NeuroEvolution: The Importance of Transfer Function Evolution and Heterogeneous Networks</a>, which also gives a short summary of neuroevolution techniques, provides an alternative approach to NEAT. It uses <a href="https://web.archive.org/web/20160730231107/http://www.cartesiangp.co.uk/" rel="nofollow noreferrer">Cartesian Genetic Programming</a> to evolve multiple activation functions.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2016-08-06T06:33:58.880Z
Answers.LastActivityDate: 2021-12-25T11:11:47.567Z
Answers.LastEditDate: 2021-12-25T11:11:47.567Z
Answers.UserDisplayName: NietzscheanAI
Answers.Body: <p>I think in this case, you'll probably want to use a genetic algorithm to generate a topology rather than working on your own. I personally like <a href="http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf" rel="nofollow noreferrer">NEAT (NeuroEvolution of Augmenting Topologies)</a>.</p>
  
  <p>The original NEAT paper involves evolving weights for connections, but if you only want a topology, you can use a weighting algorithm instead. You can also mix activation functions if you aren't sure which to use. <a href="http://blog.otoro.net/2016/05/07/backprop-neat/" rel="nofollow noreferrer">Here</a> is an example of using backpropagation and multiple neuron types.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2016-08-06T05:40:41.717Z
Answers.LastActivityDate: 2019-07-07T21:18:25.820Z
Answers.LastEditDate: 2019-07-07T21:18:25.820Z
Answers.UserDisplayName: DanTheMan
Body: <p>Assume that I want to solve an issue with a neural network that either I can't fit to existing architectures (perceptron, Konohen, etc) or I'm simply not aware of the existence of those or I'm unable to understand their mechanics and I rely on my own instead.</p>
  <p>How can I automate the choice of the architecture/topology (that is, the number of layers, the type of activations, the type and direction of the connections, etc.) of a neural network for an arbitrary problem?</p>
  <p>I'm a beginner, yet I realized that in some architectures (or, at least, in perceptrons) it is very hard if not impossible to understand the inner mechanics, as the neurons of the hidden layers don't express any mathematically meaningful context.</p>
  
Tags: <neural-networks><reference-request><hyperparameter-optimization><architecture><neuroevolution>
Title: How can I automate the choice of the architecture of a neural network for an arbitrary problem?
UserDisplayName: Zoltán Schmidt
ThreadText: # How can I automate the choice of the architecture of a neural network for an arbitrary problem?
  
  (Asked by: username_0 on 2016-08-05)
  
  Assume that I want to solve an issue with a neural network that either I can't fit to existing architectures (perceptron, Konohen, etc) or I'm simply not aware of the existence of those or I'm unable to understand their mechanics and I rely on my own instead.
  
  How can I automate the choice of the architecture/topology (that is, the number of layers, the type of activations, the type and direction of the connections, etc.) of a neural network for an arbitrary problem?
  
  I'm a beginner, yet I realized that in some architectures (or, at least, in perceptrons) it is very hard if not impossible to understand the inner mechanics, as the neurons of the hidden layers don't express any mathematically meaningful context.
  
  **Tags:** neural-networks, reference-request, hyperparameter-optimization, architecture, neuroevolution
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2016-08-06. Score: 11 | Accepted answer
  
  I think in this case, you'll probably want to use a genetic algorithm to generate a topology rather than working on your own. I personally like [NEAT (NeuroEvolution of Augmenting Topologies)](http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf).
  
  The original NEAT paper involves evolving weights for connections, but if you only want a topology, you can use a weighting algorithm instead. You can also mix activation functions if you aren't sure which to use. [Here](http://blog.otoro.net/2016/05/07/backprop-neat/) is an example of using backpropagation and multiple neuron types.
  
  
  ---
  
  ### Answer by username_1 on 2016-08-06. Score: 5
  
  [The other answer](https://ai.stackexchange.com/a/1399/2444) mentions [NEAT](https://www.cs.ucf.edu/%7Ekstanley/neat.html) to generate network weights or topologies. The paper [NeuroEvolution: The Importance of Transfer Function Evolution and Heterogeneous Networks](http://doc.gold.ac.uk/aisb50/AISB50-S11/AISB50-S11-Turner-paper.pdf), which also gives a short summary of neuroevolution techniques, provides an alternative approach to NEAT. It uses [Cartesian Genetic Programming](https://web.archive.org/web/20160730231107/http://www.cartesiangp.co.uk/) to evolve multiple activation functions.
  
  
  ---
  
  