Answers.Body: <p>The gradient that we use to train neural networks is the gradient of the loss function with respect to the parameters of each layer.</p>
  <p>The parameters usually form a very large vector, concatenating the parameters of each layer, and you compute gradients with respect to that large vector.</p>
  <p>Then to train a single layer, you just take the gradient of the loss with respect to the parameters of that layer only. Mathematically it is very simple.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-12T18:00:15.423Z
Answers.LastActivityDate: 2022-11-12T18:00:15.423Z
Answers.UserDisplayName: Dr. Snoopy
Answers.Body: <p>The way to &quot;train individual layers&quot; is to fix the weights of all other layers during training.  This <a href="https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter08_intro-to-dl-for-computer-vision.ipynb#scrollTo=gCurjG_9K9Du" rel="nofollow noreferrer">notebook</a> describes an implementation of how to fix specific layers of model weights (see section <em>Fine-tuning a pretrained model, Freezing all layers until the fourth from the last</em>).  This example is for a convolutional neural network, but the same approach can be applied to transformers.  Those weights become non-trainable, meaning that during training they cannot be updated.  Only the weights of the layer(s) of interest will remain trainable (and updatable).</p>
  <p>This is useful when the model was previously trained on other data (pre-trained model), and while you do not want to undo all of that training (i.e., you don't want to change the weights that were previously learned), you do want to refine the model with new data that you have.  Allowing weights of just some of the layers to be updated during training is one way of doing that.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-14T12:55:02.623Z
Answers.LastActivityDate: 2022-11-14T12:55:02.623Z
Answers.UserDisplayName: Snehal Patel
Body: <p>I have been reading some papers recently (example: <a href="https://arxiv.org/pdf/2012.00363.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/2012.00363.pdf</a>) which seem to be training individual layers of, say, a transformer, holding the rest of the model frozen/constant. In the case of the paper I read, this was done in an attempt to minimize parameter changes so as to reduce knowledge &quot;lost&quot; by a model when it is updated for new information.</p>
  <p>My question is, how are individual layers of a transformer trained? Like, if we run the transformer and get a gradient, how can we use that gradient to train, say, the first layer, without affecting the rest of the layers at all?</p>
  
Tags: <neural-networks><natural-language-processing><gradient-descent><transformer>
Title: How are gradients of individual layers computed?
UserDisplayName: nlp4892
ThreadText: # How are gradients of individual layers computed?
  
  (Asked by: username_0 on 2022-11-12)
  
  I have been reading some papers recently (example: <https://arxiv.org/pdf/2012.00363.pdf>) which seem to be training individual layers of, say, a transformer, holding the rest of the model frozen/constant. In the case of the paper I read, this was done in an attempt to minimize parameter changes so as to reduce knowledge "lost" by a model when it is updated for new information.
  
  My question is, how are individual layers of a transformer trained? Like, if we run the transformer and get a gradient, how can we use that gradient to train, say, the first layer, without affecting the rest of the layers at all?
  
  **Tags:** neural-networks, natural-language-processing, gradient-descent, transformer
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2022-11-12. Score: 1
  
  The gradient that we use to train neural networks is the gradient of the loss function with respect to the parameters of each layer.
  
  The parameters usually form a very large vector, concatenating the parameters of each layer, and you compute gradients with respect to that large vector.
  
  Then to train a single layer, you just take the gradient of the loss with respect to the parameters of that layer only. Mathematically it is very simple.
  
  
  ---
  
  ### Answer by username_2 on 2022-11-14. Score: 0
  
  The way to "train individual layers" is to fix the weights of all other layers during training. This [notebook](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter08_intro-to-dl-for-computer-vision.ipynb#scrollTo=gCurjG_9K9Du) describes an implementation of how to fix specific layers of model weights (see section *Fine-tuning a pretrained model, Freezing all layers until the fourth from the last*). This example is for a convolutional neural network, but the same approach can be applied to transformers. Those weights become non-trainable, meaning that during training they cannot be updated. Only the weights of the layer(s) of interest will remain trainable (and updatable).
  
  This is useful when the model was previously trained on other data (pre-trained model), and while you do not want to undo all of that training (i.e., you don't want to change the weights that were previously learned), you do want to refine the model with new data that you have. Allowing weights of just some of the layers to be updated during training is one way of doing that.
  
  
  ---
  
  