I’m working on a modification to a pytorch optimizer file. For simplicity sake, let’s take SGD(sgd.py). If I wanted to change the “for loop” on line 94 to:
for i, p in enumerate(group['params']:
And add something starting on line 111 in the for loop like:
if i==0:
  bias=torch.rand(10)
  group['params'][i+1]=bias

Also, suppose that the number of biases are 8 in this layer before going through this ‘optimizer’. Currently, it doesn’t update the parameters when I run this. What would I need to change in order to make the tensor size change?
P.S. The example above is not the full code but is just for example.
Update: So the issue isn’t the selector. It just isn’t passing the new parameters through regardless.