Yes, I have tried doing that as well. From the sample code, you can also just add “torch.cuda.empty_cache()” after the two “del track[0]” if you want to test it.
As per my current understanding, if the gpu memory is indeed cached, the allocated memory will not increase since new tensors can acquire the unused cached memory. However, in the sample output above, you can see that the allocated gpu memory keeps increasing (if you keep running it for larger number of steps, can even get Cuda out of memory).