Solved by ptrblck in post #2 
              
              
                Could you check the current memory usage on the device via nvidia-smi and make sure that no other processes are running? 
Note that besides the tensor you would need to allocate the CUDA context on the device, which might take a few hundred MBs.