Answers.Body: <p>I believe the gradients in the three layers are computed independently.</p>
  <p>They mention minimizing each layer only with respect to the weights used in that layer.  If they performed some sort of backpropagation between layers, they would be minimizing the term for one layer with respect to the weights for another layer.  Since the explicitly state that equation 1 (the loss for each layer) is simply minimized for all three layers simultaneously, they clearly imply that only the weights for that layer are considered in the implementation.</p>
  <p>But this only confuses me further as to <a href="https://ai.stackexchange.com/questions/33935/why-is-training-all-layers-at-a-time-effective-for-a-multi-layer-autoencoder">why they train all three layers simultaneously, as asked in my second question</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-19T13:19:31.570Z
Answers.LastActivityDate: 2022-03-19T13:19:31.570Z
Answers.UserDisplayName: Josiah Yoder
Body: <p><a href="https://icml.cc/2012/papers/73.pdf" rel="nofollow noreferrer">Le et al. 2013</a>'s non-weight sharing CNN has inspired me to ask <a href="https://ai.stackexchange.com/questions/6298/do-le-et-al-2012-train-all-three-autoencoder-layers-at-a-time-or-just-one">two</a> <a href="https://ai.stackexchange.com/questions/33935/why-is-training-all-layers-at-a-time-effective-for-a-multi-layer-autoencoder">questions</a> on this site previously.</p>
  <p>When training the three-layer autoencoder, do they compute dL/dW (where L is equation 1) independently in each layer or perform backpropagation from one layer to another?</p>
  
Tags: <autoencoders><multilayer-perceptrons>
Title: Do Quo et al (2013) perform backpropagation between layers?
UserDisplayName: Josiah Yoder
ThreadText: # Do Quo et al (2013) perform backpropagation between layers?
  
  (Asked by: username_0 on 2022-03-19)
  
  [Le et al. 2013](https://icml.cc/2012/papers/73.pdf)'s non-weight sharing CNN has inspired me to ask [two](https://ai.stackexchange.com/questions/6298/do-le-et-al-2012-train-all-three-autoencoder-layers-at-a-time-or-just-one) [questions](https://ai.stackexchange.com/questions/33935/why-is-training-all-layers-at-a-time-effective-for-a-multi-layer-autoencoder) on this site previously.
  
  When training the three-layer autoencoder, do they compute dL/dW (where L is equation 1) independently in each layer or perform backpropagation from one layer to another?
  
  **Tags:** autoencoders, multilayer-perceptrons
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2022-03-19. Score: 0
  
  I believe the gradients in the three layers are computed independently.
  
  They mention minimizing each layer only with respect to the weights used in that layer. If they performed some sort of backpropagation between layers, they would be minimizing the term for one layer with respect to the weights for another layer. Since the explicitly state that equation 1 (the loss for each layer) is simply minimized for all three layers simultaneously, they clearly imply that only the weights for that layer are considered in the implementation.
  
  But this only confuses me further as to [why they train all three layers simultaneously, as asked in my second question](https://ai.stackexchange.com/questions/33935/why-is-training-all-layers-at-a-time-effective-for-a-multi-layer-autoencoder).
  
  
  ---
  
  