not always. at some point it doesn’t go into the inner loop and uses the loss from the last outer loop iteration in loss.backward(). That’s what I think is happening that corresponds to the error message.