Answers.Body: <p>See <a href="https://arxiv.org/pdf/1707.09725.pdf" rel="nofollow noreferrer">Analysis and Optimization of Convolutional Neural Network Architectures</a> (my master's thesis).</p>
  
  <blockquote>
    <ol>
    <li>Is there a way that helps me to architect my model fundamentally before training?</li>
    </ol>
  </blockquote>
  
  <p>Yes, but the architecture learning approaches are computationally intensive. When I wrote my masters thesis it was ~250,000 USD to run the Google experiment. Meanwhile, there seem to be more efficient methods, e.g. <a href="https://autokeras.com/" rel="nofollow noreferrer">https://autokeras.com/</a></p>
  
  <p>See Chapter 3.</p>
  
  <blockquote>
    <ol start="2">
    <li>Once I train model how do I know which among these variables (number of filters, size, number of convolutional layers, FC layers) should be changed - increased or decreased?</li>
    </ol>
  </blockquote>
  
  <p>See Chapter 2.5 for some approaches. But there is no silver bullet / no clear answer to this question.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-23T19:39:49.210Z
Answers.LastActivityDate: 2020-05-19T20:12:28.373Z
Answers.LastEditDate: 2020-05-19T20:12:28.373Z
Answers.UserDisplayName: Martin Thoma
Body: <p>While we train a CNN model we often experiment with the number of filters, the number of convolutional layers, FC layers, filter size, sometimes stride, activation function, etc. More often than not after training the model once, it is just a trial &amp; error process.  </p>
  
  <ol>
  <li><p>Is there a way that helps me to architect my model fundamentally before training?</p></li>
  <li><p>Once I train model, how do I know which among these variables (number of filters, size, number of convolutional layers, FC layers) should be
  changed - increased or decreased?</p></li>
  </ol>
  
  <p>P.S. This question assumes that data is sufficient in volume and annotated properly and still accuracy is not up to the mark. So, I've ruled out the possibility of non-architectural flaws for the question.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-09-23T21:26:40.097Z
Comments.Text: Don't forget the possibility that the relationship between the data and target is not strong in the first place. CNNs are constrained by reality. If you have a really great labeled dataset of cat pictures and traits of their owner, it is fairly unlikely you could perform better than statistical average of labels when tryin to infer e.g. the cat's owner's shoe size from a picture of the cat.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-09-24T21:31:42.917Z
Comments.Text: @NeilSlater you are right but in my case I have ruled out the possibility of non architectural flaw for the ease of the answer. However you are very right :)
Comments.UserDisplayName: Ruchit Dalwadi
Tags: <neural-networks><machine-learning><deep-learning><convolutional-neural-networks><ai-design>
Title: Is there a way that helps me to architect my CNN fundamentally before training?
UserDisplayName: Ruchit Dalwadi
ThreadText: # Is there a way that helps me to architect my CNN fundamentally before training?
  
  (Asked by: username_0 on 2018-09-22)
  
  While we train a CNN model we often experiment with the number of filters, the number of convolutional layers, FC layers, filter size, sometimes stride, activation function, etc. More often than not after training the model once, it is just a trial & error process.
  
  1. Is there a way that helps me to architect my model fundamentally before training?
  2. Once I train model, how do I know which among these variables (number of filters, size, number of convolutional layers, FC layers) should be
     changed - increased or decreased?
  
  P.S. This question assumes that data is sufficient in volume and annotated properly and still accuracy is not up to the mark. So, I've ruled out the possibility of non-architectural flaws for the question.
  
  **Tags:** neural-networks, machine-learning, deep-learning, convolutional-neural-networks, ai-design
  
  > **Comments:**
  > **username_1**: Don't forget the possibility that the relationship between the data and target is not strong in the first place. CNNs are constrained by reality. If you have a really great labeled dataset of cat pictures and traits of their owner, it is fairly unlikely you could perform better than statistical average of labels when tryin to infer e.g. the cat's owner's shoe size from a picture of the cat.  
  > **username_0**: @username_1 you are right but in my case I have ruled out the possibility of non architectural flaw for the ease of the answer. However you are very right :)  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2018-09-23. Score: 2 | Accepted answer
  
  See [Analysis and Optimization of Convolutional Neural Network Architectures](https://arxiv.org/pdf/1707.09725.pdf) (my master's thesis).
  
  > 1. Is there a way that helps me to architect my model fundamentally before training?
  
  Yes, but the architecture learning approaches are computationally intensive. When I wrote my masters thesis it was ~250,000 USD to run the Google experiment. Meanwhile, there seem to be more efficient methods, e.g. <https://autokeras.com/>
  
  See Chapter 3.
  
  > 2. Once I train model how do I know which among these variables (number of filters, size, number of convolutional layers, FC layers) should be changed - increased or decreased?
  
  See Chapter 2.5 for some approaches. But there is no silver bullet / no clear answer to this question.
  
  
  ---
  
  