I’m not sure which difference you are pointing to. But the issue sounds as if you are storing the computation graph in each iterations, which could slow down your code and would also be visible in an increasing memory usage.


No, torch.cuda.empty_cache() will potentially only slow down your code further since synchronizing cudaMalloc calls would be needed to re-allocate the memory, and won’t save any memory.