Yes, you can put it out of the inside loop, and calculate the loss to other tensor. But it is not nesessary because such action will introduce a little complex backward process, and you would get no improvement in performance. So you could just take the optimizer.step() out of the inside loop.
In additianal, this way will cause an another result different to origin code version for optimizer.step() executed at every step in inside loop. Which is the best one relies on the background problem.
I am sorry for lated answor.