@Gopal_Sharma I can see why the two approaches are identical mathematically, but what is the difference computationally?
If I understand correctly, in the first case, every iteration extends the graph  (in the loss = loss + criterion(...) line) but the backward() function is then only called once per minibatch, while in the second version, the graph is always the same, but backward() gets called on every example in the minibatch.
So which of the two solutions would be preferable and why? I am not sure I understand how much bigger the graph would need to get in the first version and which parts of the graph would need to be kept around until zer_grad is called again. But I suppose it depends on the relative cost of this versus calling backward()?