As far as I understand, torch.autograd. backward() calculates the  sum of gradients of the tensors w.r.t. graph leaves. And optimizer.step() calculates the new parameter(weights) values based on these gradients.