I cannot find a place in a source code for optim.SGD where the actual weight update takes place. I expect to see “something like”
model.layer.weight -= lr * model.layer.weight.grad
model.layer.bias -= lr * model.layer.bias.grad

but I cannot find it…
I checked torch/optim/sgd.py and there is no obvious weight update …
Help will be very appreciated!