from the surface of your code snippet, it looks fine, so I’m not sure where the memory growth / holding onto the graph is coming from.
A common pattern that is dangerously growing the graph across iterations is:
total_loss += loss # for reporting / logging purposes

here, loss is a torch zero-dimensional Tensor, so it records gradient (and hence holds onto the graph).
The right thing to do would be:
total_loss += loss.item() # loss.item() converts it into a python number