Solved by KFrank in post #2 
              
              
                Hi Afsaneh! 

The short answer is to set the specific element in question back to 
1 after calling opt.step(). 
As you’ve recognized, requires_grad applies to an entire tensor, 
not separately to individual elements.  It is also true that an Optimizer 
applies to entire tensors, and not to individua…