Answers.Body: <p>You can freeze a model's parameters in pytorch using the following code:</p>
  <pre><code>for param in encoder_model.parameters():
      param.requires_grad = False
  </code></pre>
  <p>or you can simply not pass the encoder's parameters to the optimizer</p>
  <pre><code>optim = optim.Adam(generator_model.parameters())
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-10T19:06:58.640Z
Answers.LastActivityDate: 2024-01-10T19:06:58.640Z
Answers.UserDisplayName: Dennis Yang
Body: <p>These are components in my model:</p>
  <ul>
  <li>A generator</li>
  <li>An encoder: a pretrained, and should not updated.</li>
  <li>A loss function.</li>
  </ul>
  <p>Input is passed to the encoder to generate X, X is then passed to generator to generated a value Y. Y is passed to the encoder to get Z. The loss function computes error between Z and X. I detached X to prevent update the encoder. But I cannot detach Z, otherwise, the generator cannot be updated. But I also realize if Z is not detached then the encoder is also updated.
  In the case, how can I prevent the encoder being updated, and still allow the generator get feedback from the loss?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-01-09T11:52:16.243Z
Comments.Text: It depends on the framework you use
Comments.UserDisplayName: Aray Karjauv
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-01-09T23:45:45.027Z
Comments.Text: I used pytorch, and could you explain how it depends on the framework?
Comments.UserDisplayName: Jesse
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-01-10T21:17:09.967Z
Comments.Text: according to the rules of this community, programming questions are not allowed, so you better ask your question on StackOverflow. but there are a couple of ways to achieve it. You can disable backpropagation for specific layers or you can call `loss.backward()` to calc the gradients and `optimizer.step()` to update the weights for a specific part of your model
Comments.UserDisplayName: Aray Karjauv
Tags: <deep-learning><backpropagation><optimization>
Title: How to prevent update a pretrained model if a model is optimized with backpropagation?
UserDisplayName: Jesse
ThreadText: # How to prevent update a pretrained model if a model is optimized with backpropagation?
  
  (Asked by: username_0 on 2024-01-09)
  
  These are components in my model:
  
  * A generator
  * An encoder: a pretrained, and should not updated.
  * A loss function.
  
  Input is passed to the encoder to generate X, X is then passed to generator to generated a value Y. Y is passed to the encoder to get Z. The loss function computes error between Z and X. I detached X to prevent update the encoder. But I cannot detach Z, otherwise, the generator cannot be updated. But I also realize if Z is not detached then the encoder is also updated.
  In the case, how can I prevent the encoder being updated, and still allow the generator get feedback from the loss?
  
  **Tags:** deep-learning, backpropagation, optimization
  > **Comments:**
  > **username_1**: It depends on the framework you use  
  > **username_0**: I used pytorch, and could you explain how it depends on the framework?  
  > **username_1**: according to the rules of this community, programming questions are not allowed, so you better ask your question on StackOverflow. but there are a couple of ways to achieve it. You can disable backpropagation for specific layers or you can call `loss.backward()` to calc the gradients and `optimizer.step()` to update the weights for a specific part of your model  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2024-01-10. Score: 1
  
  You can freeze a model's parameters in pytorch using the following code:
  for param in encoder_model.parameters():
      param.requires_grad = False
  
  or you can simply not pass the encoder's parameters to the optimizer
  optim = optim.Adam(generator_model.parameters())
  
  
  ---
  
  