Solved by ptrblck in post #3 
              
              
                You are creating a new .require_grad attribute for each parameter, while you most likely want to set the .requires_grad attribute to False for the parameters (note the missing s).