Hi @oezguensi. Thanks for your reply. Yes that solves the problem. But in this case, I still need to do loss.backward() and perform backprop for all the weights. Then, I can do the update as you specified. My main question was about not performing backprop for some of the weights within the layer. I really appreciate your help though!