ngimel:

Couple hundred MB are usually taken just by initializing cuda. Look at your memory consumption after
a=torch.cuda.FloatTensor(1)

, that would give you the framework overhead.


Thanks for this suggestion.
Any thoughts as to why this overhead might be a lot more than a couple hundred MB?
I checked this now exactly as you suggested with allocating a unit sized tensor, and in my case it seems to be 1229 [MB]! This is clearly too much.
I checked right before the allocation and it was close to zero, and just after allocating this unit tensor, it jumped to 1229 [MB].
I’m using Pytorch v1.7, Cuda 10.1 and Tesla V100 GPU.