Solved by rahulvigneswaran in post #4 
              
              
                The answer to this question was answered on stackoverflow by a user named jodag. I have modified and added it here. 
SOLUTION: 
What you have seems like it would work provided you did it after  loss.backward()  and before  optimizer.step()  (referring to the common usage for these variable names). T…