Answers.Body: <p>Yes, you can fix (or freeze) some of the weights during the training of a neural network. In fact, this is done in the most common form of <em>transfer learning</em> (which is described <a href="https://www.tensorflow.org/tutorials/images/transfer_learning" rel="nofollow noreferrer">here</a>). I don't know exactly how this affects learning in general. In transfer learning, this is definitely beneficial, as we are freezing the weights that are associated with the learned <em>general</em> features of objects, such as corners (where <em>general</em> here is defined intuitively), which can be useful for other tasks, so, by having them frozen, we reuse them.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-07-06T01:43:05.483Z
Answers.LastActivityDate: 2021-07-06T15:43:06.233Z
Answers.LastEditDate: 2021-07-06T15:43:06.233Z
Answers.UserDisplayName: nbro
Answers.Body: <p>You may have more success freezing weights in the central corridor rather than in layers.  The central corridor is the middle of all the layers.  <a href="https://medium.com/@mbonsign/training-neural-networks-a-new-perspective-on-core-stability-and-peripheral-learning-5bd7bf3a7fce" rel="nofollow noreferrer">https://medium.com/@mbonsign/training-neural-networks-a-new-perspective-on-core-stability-and-peripheral-learning-5bd7bf3a7fce</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-10-31T01:03:56.420Z
Answers.LastActivityDate: 2024-10-31T01:03:56.420Z
Answers.UserDisplayName: Micheal Bee
Body: <p>Is it possible to exclude specific layers from the optimization?</p>
  <p>For example, let's say I have an input layer, 2 hidden layers, and the output layer. I know there is a perfect solution for my problem with this setup and I already know the perfect weights between the first and the second hidden layer.</p>
  <p>Can I have the weights between the first and the second hidden layer be fixed during the training phase?</p>
  <p>I understand that I could just not update these specific weights after I computed the backpropagation for the entire network. But if I throw away those specific weights, will this affect the optimization of the rest of my weights?</p>
  
Tags: <machine-learning><convolutional-neural-networks><training><backpropagation><weights>
Title: Can some of the weights be fixed during the training of a neural network?
UserDisplayName: NewToAI2021
ThreadText: # Can some of the weights be fixed during the training of a neural network?
  
  (Asked by: username_0 on 2021-07-05)
  
  Is it possible to exclude specific layers from the optimization?
  
  For example, let's say I have an input layer, 2 hidden layers, and the output layer. I know there is a perfect solution for my problem with this setup and I already know the perfect weights between the first and the second hidden layer.
  
  Can I have the weights between the first and the second hidden layer be fixed during the training phase?
  
  I understand that I could just not update these specific weights after I computed the backpropagation for the entire network. But if I throw away those specific weights, will this affect the optimization of the rest of my weights?
  
  **Tags:** machine-learning, convolutional-neural-networks, training, backpropagation, weights
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2021-07-06. Score: 4
  
  Yes, you can fix (or freeze) some of the weights during the training of a neural network. In fact, this is done in the most common form of *transfer learning* (which is described [here](https://www.tensorflow.org/tutorials/images/transfer_learning)). I don't know exactly how this affects learning in general. In transfer learning, this is definitely beneficial, as we are freezing the weights that are associated with the learned *general* features of objects, such as corners (where *general* here is defined intuitively), which can be useful for other tasks, so, by having them frozen, we reuse them.
  
  
  ---
  
  ### Answer by username_2 on 2024-10-31. Score: 0
  
  You may have more success freezing weights in the central corridor rather than in layers. The central corridor is the middle of all the layers. <https://medium.com/@mbonsign/training-neural-networks-a-new-perspective-on-core-stability-and-peripheral-learning-5bd7bf3a7fce>.
  
  
  ---
  
  