Well, if you want the gradients of the first backward to be used in the update, then you shouldn’t run zero_grad after it.
When you run two backward passes one after the other, the calculated grads get added together.