It is usually that the graph is reused.
So either that you do some differentiable computation outside the loop, or that you re-use results from one iteration to the next.
I would usually try to get a repro as small as possible. That helps identify the potential faulty Tensors as mentioned above.
If that doesn’t help, you can use something like torchviz 22 to print the graph and see where the one from the second iteration is attached to the one in the first.