After reading several posts, I’ve come to this same conclusion. Which leads to my next question, here:
  
    
    
    Modifying/Custom Optimizer: Updating Layer Values 
  
  
    I’m working on a modification to a pytorch optimizer file. For simplicity sake, let’s take SGD(sgd.py). If I wanted to change the “for loop” on line 94 to: 
for i, p in enumerate(group['params']: 
And add something starting on line 111 in the for loop like: 
if i==0:
  bias=torch.rand(10)
  group['params'][i+1]=bias

Currently, it doesn’t update the values when I run this. What would I need to change in order to make the next tensor update its values? 
P.S. The example above is not the full code…