Wondering what can cause increase in cache memory?
In my code, torch.cuda.max_memory_allocated(0) shows only 3GB but nvidia-smi shows 10GB. I expected some cache but not 3 times the real memory. How should I investigate and reduce that?
I think empty_cache will slow down the algorithm, right?