<p>See <a href="https://arxiv.org/pdf/1707.09725.pdf" rel="nofollow noreferrer">Analysis and Optimization of Convolutional Neural Network Architectures</a> (my master's thesis).</p>

<blockquote>
  <ol>
  <li>Is there a way that helps me to architect my model fundamentally before training?</li>
  </ol>
</blockquote>

<p>Yes, but the architecture learning approaches are computationally intensive. When I wrote my masters thesis it was ~250,000 USD to run the Google experiment. Meanwhile, there seem to be more efficient methods, e.g. <a href="https://autokeras.com/" rel="nofollow noreferrer">https://autokeras.com/</a></p>

<p>See Chapter 3.</p>

<blockquote>
  <ol start="2">
  <li>Once I train model how do I know which among these variables (number of filters, size, number of convolutional layers, FC layers) should be changed - increased or decreased?</li>
  </ol>
</blockquote>

<p>See Chapter 2.5 for some approaches. But there is no silver bullet / no clear answer to this question.</p>

CC BY-SA 4.0
2018-09-23T19:39:49.210Z
2020-05-19T20:12:28.373Z
2020-05-19T20:12:28.373Z
Martin Thoma
<p>While we train a CNN model we often experiment with the number of filters, the number of convolutional layers, FC layers, filter size, sometimes stride, activation function, etc. More often than not after training the model once, it is just a trial &amp; error process.  </p>

<ol>
<li><p>Is there a way that helps me to architect my model fundamentally before training?</p></li>
<li><p>Once I train model, how do I know which among these variables (number of filters, size, number of convolutional layers, FC layers) should be
changed - increased or decreased?</p></li>
</ol>

<p>P.S. This question assumes that data is sufficient in volume and annotated properly and still accuracy is not up to the mark. So, I've ruled out the possibility of non-architectural flaws for the question.</p>

CC BY-SA 4.0
2018-09-23T21:26:40.097Z
Don't forget the possibility that the relationship between the data and target is not strong in the first place. CNNs are constrained by reality. If you have a really great labeled dataset of cat pictures and traits of their owner, it is fairly unlikely you could perform better than statistical average of labels when tryin to infer e.g. the cat's owner's shoe size from a picture of the cat.
Neil Slater
CC BY-SA 4.0
2018-09-24T21:31:42.917Z
@NeilSlater you are right but in my case I have ruled out the possibility of non architectural flaw for the ease of the answer. However you are very right :)
Ruchit Dalwadi
<neural-networks><machine-learning><deep-learning><convolutional-neural-networks><ai-design>
Is there a way that helps me to architect my CNN fundamentally before training?
Ruchit Dalwadi
# Is there a way that helps me to architect my CNN fundamentally before training?

(Asked by: username_0 on 2018-09-22)

While we train a CNN model we often experiment with the number of filters, the number of convolutional layers, FC layers, filter size, sometimes stride, activation function, etc. More often than not after training the model once, it is just a trial & error process.

1. Is there a way that helps me to architect my model fundamentally before training?
2. Once I train model, how do I know which among these variables (number of filters, size, number of convolutional layers, FC layers) should be
   changed - increased or decreased?

P.S. This question assumes that data is sufficient in volume and annotated properly and still accuracy is not up to the mark. So, I've ruled out the possibility of non-architectural flaws for the question.

**Tags:** neural-networks, machine-learning, deep-learning, convolutional-neural-networks, ai-design

> **Comments:**
> **username_1**: Don't forget the possibility that the relationship between the data and target is not strong in the first place. CNNs are constrained by reality. If you have a really great labeled dataset of cat pictures and traits of their owner, it is fairly unlikely you could perform better than statistical average of labels when tryin to infer e.g. the cat's owner's shoe size from a picture of the cat.  
> **username_0**: @NeilSlater you are right but in my case I have ruled out the possibility of non architectural flaw for the ease of the answer. However you are very right :)  

---

## 1 answers

### Answer by username_2 on 2018-09-23. Score: 2 | Accepted answer

See [Analysis and Optimization of Convolutional Neural Network Architectures](https://arxiv.org/pdf/1707.09725.pdf) (my master's thesis).

> 1. Is there a way that helps me to architect my model fundamentally before training?

Yes, but the architecture learning approaches are computationally intensive. When I wrote my masters thesis it was ~250,000 USD to run the Google experiment. Meanwhile, there seem to be more efficient methods, e.g. <https://autokeras.com/>

See Chapter 3.

> 2. Once I train model how do I know which among these variables (number of filters, size, number of convolutional layers, FC layers) should be changed - increased or decreased?

See Chapter 2.5 for some approaches. But there is no silver bullet / no clear answer to this question.


---

