Hi,
I think there are a few related threads

Update only sub-elements of weights - autograd - PyTorch Forums 1
Passing a subset of the parameters to an optimizer equivalent to setting requires_grad of subset only to True? - autograd - PyTorch Forums 2