Could you check the current memory usage on the device via nvidia-smi and make sure that no other processes are running?
Note that besides the tensor you would need to allocate the CUDA context on the device, which might take a few hundred MBs.