SimonW:

Apologies. I went ahead and check the code. The smallest block size is 1MB. So the memory usage you see is not related to that.


OK, so 1MB it is. We are on the same page now.

However, it is still very possible that most of 0.5GB is CUDA ctx. The actual size depends on the CUDA version & card type (reference 1). I checked with a 1080Ti & CUDA 8 I have access to, it takes 343MB. And from my memory on a Titan Xp and CUDA 9 it took ~200MB.

Fascinating - the larger the card the less overhead! Mine is 8GB and is taking 0.5GB.
Now I know it’s card-dependent. thank you!

Btw, in your code, do you mean to use memory_cached rather than max_memory_cached ?

Yes, you’re correct. fixed. Thank you.