Answers.Body: <p>I am not sure if there really are contradicting opinions on this matter. CNNs, RNNs, LSTMs all have specific types of data they are good at predicting. Depth and width, or in general the size of the neural network mostly depends on the size of your dataset. You don't want to build a too large network that will overfit the available data, which can usually be understood to be the case after running a few epochs but in general, number of trainable parameters should be smaller than to total number of independent data points you have to avoid memorizing the data definitely.</p>
  <p>Width and depth of the network depends on the size of the data available (as mentioned before), but can simply be thought just as another hyper-parameter that is to be decided by training itself. It can be defined as a part of cross validation process of building a neural network.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-28T11:56:24.753Z
Answers.LastActivityDate: 2021-09-28T11:56:24.753Z
Answers.UserDisplayName: serali
Body: <p>When we take a look at the literature there are so many opinions.
  I was wondering what are some generally good practices to design an architecture, like how much depth would you prefer and how much width would you prefer.
  Should the number of training data influence your decisions of designing the architecture.
  What should the number of parameters be ? etc.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-28T12:08:56.710Z
Comments.Text: Can you please provide 2-3 examples of "literature" that are contradicting each other by citing them and quoting the relevant parts?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-28T12:11:59.247Z
Comments.Text: @nbro https://stats.stackexchange.com/a/223637/272525 and https://arxiv.org/abs/1605.07146
Comments.UserDisplayName: Sadaf Shafi
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-28T12:16:55.897Z
Comments.Text: Can you please edit your post to quote (provide 2 sentences) that are contradicting themselves?
Comments.UserDisplayName: nbro
Tags: <neural-networks><convolutional-neural-networks><training>
Title: What practically makes a good architecture of ANN?
UserDisplayName: Sadaf Shafi
ThreadText: # What practically makes a good architecture of ANN?
  
  (Asked by: username_0 on 2021-09-28)
  
  When we take a look at the literature there are so many opinions.
  I was wondering what are some generally good practices to design an architecture, like how much depth would you prefer and how much width would you prefer.
  Should the number of training data influence your decisions of designing the architecture.
  What should the number of parameters be ? etc.
  
  **Tags:** neural-networks, convolutional-neural-networks, training
  
  > **Comments:**
  > **username_1**: Can you please provide 2-3 examples of "literature" that are contradicting each other by citing them and quoting the relevant parts?  
  > **username_0**: @username_1 https://stats.stackexchange.com/a/223637/272525 and https://arxiv.org/abs/1605.07146  
  > **username_1**: Can you please edit your post to quote (provide 2 sentences) that are contradicting themselves?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-09-28. Score: 1
  
  I am not sure if there really are contradicting opinions on this matter. CNNs, RNNs, LSTMs all have specific types of data they are good at predicting. Depth and width, or in general the size of the neural network mostly depends on the size of your dataset. You don't want to build a too large network that will overfit the available data, which can usually be understood to be the case after running a few epochs but in general, number of trainable parameters should be smaller than to total number of independent data points you have to avoid memorizing the data definitely.
  
  Width and depth of the network depends on the size of the data available (as mentioned before), but can simply be thought just as another hyper-parameter that is to be decided by training itself. It can be defined as a part of cross validation process of building a neural network.
  
  
  ---
  
  