Andrea_Rosasco:

For some reason empty_cache() manages to deallocate 2 MiB (this is consistent and not due to other processes on the same GPU I’ve tried it multiple times). Thinkig about it I guess that those 2 MiB are the size of the tensor I allocate.


Yes, the 2MB are shown in the torch.cuda.memory_reserved() output, which gives you the allocated and cached memory: 2097152 / 1024**2 = 2.0MB.



 Andrea_Rosasco:

The other 1001 MiB are probably allocated by the CUDA backend for some internal functioning reason.
I was gonna ask if there’s a way to prevent that but I don’t think so.


Yes, these are used by the CUDA context as described before and cannot be freed.