If the memory usage increases in each iteration, you might accidentally store the computation graph by e.g. appending the output of loss to a list without detaching it.
However, I cannot spot the error in your current code and would recommend to remove all places where tensors could be stored in some kind of container to further isolate this issue.
If the “pure” training loop also increases the memory, it might be a valid issue.