Hi Alexey and Drezal!



 Alexey_Demyanchuk:

I am not aware of the method how you can do requires_grad = False for the slice of the weights.


As Alexey notes, you can’t apply requires_grad to only part of a
tensor.


Anyway, you can zero some slice of the gradients before optimization step, so this exact slice of weights don’t changed after optimization step.


Please note that this approach – zeroing out parts of .grad before
calling opt.step() – doesn’t work in general.  Some optimizers (e.g.,
when using momentum or weight decay) will change the weights
even if .grad is zero.
The more general approach will be to copy the weights you want
frozen, call opt.step() and then rewrite the frozen weights back
into your param.
Best.
K. Frank