After you zero out your gradient and call .backward() on your loss, each parameter in your model will have a field .grad that contains its gradients. It is this field that is used by the optimizer when you do opt.step().