Hi,
The memory reported here is only the memory used by pytorch Tensors. The cuda driver will use some memory as well. And if you have other things running on the gpu (like a graphical interface) that will take some memory as well.
I think CUDA in some architectures allocates memory by blocks of 2MB. So if less than that is available, it’s possible that any allocation that needs a new block will fail.
What is the memory usage reported by nvidia-smi? Note that it’s possible that the peak memory usage is very fast and you don’t see it there.