Answers.Body: <p>Yes.  You can train end-to-end.  The introduction of convolution kernels with associated pooling layers to the sequence of forward feed operations on the signals does not change the basic principles.</p>
  
  <ul>
  <li>Gradient descent estimates the incremental change required to converge on an optimal behavior.</li>
  <li>The corrective error must be distributed, which is most efficiently done by employing the derivative of the activation function sequentially to each set of parameters (whether convolution kernels or matrices that attenuates inputs to activation vectors) from the output back to the input.</li>
  </ul>
  
  <p>Consider studying <a href="http://www.jefkine.com/general/2016/09/05/backpropagation-in-convolutional-neural-networks/" rel="nofollow noreferrer"><em>Backpropagation In Convolutional Neural Networks</em> on Jefkine.com</a>, which clarifies the application of those principles with convolution-pooling pairs.</p>
  
  <p>There is another approach, borrowing from wisdom gained in the development of analog feedback in instrumentation.  There are times when a sequence of operations can be better trained with more than one feedback loop, which requires some determination of error or wellness at intermediate stages and breaks the system into segments that each train based on those intermediate criteria.</p>
  
  <p>This other approach is hierarchical, and an overall convergence may be controlled by a higher level back propagation, considering each segment as a black box.  As in analog circuitry, multiple <strong>degrees of freedom</strong> with semi-independent convergence mechanisms has been shown to allow for deeper sequences without a major loss of convergence reliability or accuracy.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-08-12T23:00:35.787Z
Answers.LastActivityDate: 2018-08-12T23:00:35.787Z
Answers.UserDisplayName: Douglas Daseeco
Body: <p>When it comes to CNNs, I don't understand 2 things in the training process:</p>
  
  <ol>
  <li><p>How do I pass the error back when there are pooling layers between the convolutional layers?</p></li>
  <li><p>And if I know how it's done, can I train all the layers just like layers in normal Feed Forward Neural Nets?</p></li>
  </ol>
  
Tags: <convolutional-neural-networks><backpropagation>
Title: How to train a CNN
UserDisplayName: Evator
ThreadText: # How to train a CNN
  
  (Asked by: username_0 on 2018-08-12)
  
  When it comes to CNNs, I don't understand 2 things in the training process:
  
  1. How do I pass the error back when there are pooling layers between the convolutional layers?
  2. And if I know how it's done, can I train all the layers just like layers in normal Feed Forward Neural Nets?
  
  **Tags:** convolutional-neural-networks, backpropagation
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-08-12. Score: 2 | Accepted answer
  
  Yes. You can train end-to-end. The introduction of convolution kernels with associated pooling layers to the sequence of forward feed operations on the signals does not change the basic principles.
  
  * Gradient descent estimates the incremental change required to converge on an optimal behavior.
  * The corrective error must be distributed, which is most efficiently done by employing the derivative of the activation function sequentially to each set of parameters (whether convolution kernels or matrices that attenuates inputs to activation vectors) from the output back to the input.
  
  Consider studying [*Backpropagation In Convolutional Neural Networks* on Jefkine.com](http://www.jefkine.com/general/2016/09/05/backpropagation-in-convolutional-neural-networks/), which clarifies the application of those principles with convolution-pooling pairs.
  
  There is another approach, borrowing from wisdom gained in the development of analog feedback in instrumentation. There are times when a sequence of operations can be better trained with more than one feedback loop, which requires some determination of error or wellness at intermediate stages and breaks the system into segments that each train based on those intermediate criteria.
  
  This other approach is hierarchical, and an overall convergence may be controlled by a higher level back propagation, considering each segment as a black box. As in analog circuitry, multiple **degrees of freedom** with semi-independent convergence mechanisms has been shown to allow for deeper sequences without a major loss of convergence reliability or accuracy.
  
  
  ---
  
  