In batch learning, the weights are only updated once per epoch, so the optimizer.step() call needs to be at the end of the outer loop.  I’m starting to think that the optimizer.zero_grad() call should be just before the inner loop.