James_e:

How does this work? Would the second .backward() call not overwrite the gradient values stored in .grad attribute of the optimiser?


No, each backward call accumulates the gradients in the .grad attribute of the used parameters, which is also why you have to call optimizer/model.zero_grad() before calculating the gradients of a new iteration.