Because if you save 1 variable, you are saving the entire computation graph
Even with cpu()
Maybe try detach()
Double check if you are keeping anything at each iteration. You can also put the inner of the for loop into a function, so when it’s out of context, it’ll auto free all memory (except what you return of course)