Solved by Alexey_Demyanchuk in post #2 
              
              
                Hey. It is not 100% clear for me what are you trying to achieve, but I answer as far as I understand. 
Pytorch weights tensors all have attribute requires_grad. If set to False weights of this ‘layer’ will not be updated during optimization process, simply frozen. 
You can do it in this manner, all …