Answers.Body: <ol>
  <li><p>Input layers will always have the dimensionality of your input data(for every model I can think of).</p></li>
  <li><p>See above, the deviation between hidden layers can be significant. For example, 128 in the first hidden and 64 in the rest(or vice versa).</p></li>
  <li><p>This question in particular will always be problem dependent. It is decided via architecture search or intuition/experience combined with some exploratory search.</p></li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-06T16:18:45.690Z
Answers.LastActivityDate: 2019-08-06T20:06:18.673Z
Answers.LastEditDate: 2019-08-06T20:06:18.673Z
Answers.UserDisplayName: hisairnessag3
Answers.Body: <p>There is no right answer to this question. But, I would like to point you to <a href="https://stats.stackexchange.com/questions/181/how-to-choose-the-number-of-hidden-layers-and-nodes-in-a-feedforward-neural-netw">an answer on CV</a> that addresses the <em>mean</em> of your problem.</p>
  <p>Two points from the accepted answer that I want to draw your attention to are:</p>
  <blockquote>
  <p>a) There are some empirically-derived rules-of-thumb, of these, the most commonly relied on is <em>'the optimal size of the hidden layer is usually between the size of the input and size of the output layers'</em>.</p>
  <p>b) In sum, for most problems, one could probably get decent performance (even without a second optimization step) by setting the hidden layer configuration using just two rules: (i) number of hidden layers equals one; and (ii) the number of neurons in that layer is the mean of the neurons in the input and output layers.</p>
  </blockquote>
  <p>Other answers in the thread are also very insightful. I will recommend you to go through the answers and figure out the standard deviation and just assume things are normal, and hence you would have your distribution.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-06T18:00:51.283Z
Answers.LastActivityDate: 2019-08-06T18:16:51.990Z
Answers.LastEditDate: 2020-06-17T09:57:20.710Z
Answers.UserDisplayName: naive
Body: <p><strong>In a neural network, by how much does the number of neurons typically vary from layer to layer?</strong></p>
  <p>Note that I am <strong>NOT</strong> asking how to find the optimal number of neurons per layer.</p>
  <p>As a hardware design engineer with no practical experience programming neural networks, I would like to glean for example</p>
  <ol>
  <li><p>By how much does the number of neurons in hidden layers typically vary from that of the input layer?</p>
  </li>
  <li><p>What is the maximum deviation in the number of hidden layer neurons to the number of input layer neurons?</p>
  </li>
  <li><p>How commonly do you see a large spike in the number of neurons?</p>
  </li>
  </ol>
  <p>It likely depends on the application so I would like to hear from as many people as possible. Please tell me about your experience.</p>
  
Tags: <neural-networks><deep-learning><hyper-parameters><artificial-neuron><layers>
Title: In a neural network, by how much does the number of neurons typically vary from layer to layer?
UserDisplayName: Angela Johnson
ThreadText: # In a neural network, by how much does the number of neurons typically vary from layer to layer?
  
  (Asked by: username_0 on 2019-08-06)
  
  **In a neural network, by how much does the number of neurons typically vary from layer to layer?**
  
  Note that I am **NOT** asking how to find the optimal number of neurons per layer.
  
  As a hardware design engineer with no practical experience programming neural networks, I would like to glean for example
  
  1. By how much does the number of neurons in hidden layers typically vary from that of the input layer?
  2. What is the maximum deviation in the number of hidden layer neurons to the number of input layer neurons?
  3. How commonly do you see a large spike in the number of neurons?
  
  It likely depends on the application so I would like to hear from as many people as possible. Please tell me about your experience.
  
  **Tags:** neural-networks, deep-learning, hyper-parameters, artificial-neuron, layers
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2019-08-06. Score: 1
  
  1. Input layers will always have the dimensionality of your input data(for every model I can think of).
  2. See above, the deviation between hidden layers can be significant. For example, 128 in the first hidden and 64 in the rest(or vice versa).
  3. This question in particular will always be problem dependent. It is decided via architecture search or intuition/experience combined with some exploratory search.
  
  
  ---
  
  ### Answer by username_2 on 2019-08-06. Score: 0
  
  There is no right answer to this question. But, I would like to point you to [an answer on CV](https://stats.stackexchange.com/questions/181/how-to-choose-the-number-of-hidden-layers-and-nodes-in-a-feedforward-neural-netw) that addresses the *mean* of your problem.
  
  Two points from the accepted answer that I want to draw your attention to are:
  
  > a) There are some empirically-derived rules-of-thumb, of these, the most commonly relied on is *'the optimal size of the hidden layer is usually between the size of the input and size of the output layers'*.
  >
  > b) In sum, for most problems, one could probably get decent performance (even without a second optimization step) by setting the hidden layer configuration using just two rules: (i) number of hidden layers equals one; and (ii) the number of neurons in that layer is the mean of the neurons in the input and output layers.
  
  Other answers in the thread are also very insightful. I will recommend you to go through the answers and figure out the standard deviation and just assume things are normal, and hence you would have your distribution.
  
  
  ---
  
  