I’m trying to setup torch 1.1 with CUDA, but I’m encountering random errors difficult to debug. After installation, when I try to run the minimal code to test CUDA functionality:
import torch
torch.zeros(5).cuda()

I get random cryptic errors, one example being
fatal   : Memory allocation failure
fatal   : Memory allocation failure
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: CUDA error: unknown error

or a CUDA out of memory error (even tough I’m only trying to copy a small array) or even a segmentation fault, causing Python to crash.
To give some context, I’m using a conda environment with Python 3.7, and I have a CUDA 9.0 installation with cuDNN 7.5.1. I install PyTorch with the command from the website:
conda install pytorch torchvision cudatoolkit=9.0 -c pytorch

I have also tried to using Pytorch 0.4 and CUDA 10.0/10.1, but the error persists. The installed driver version is 418.74. Finally, I do not have root access since I’m computing on a shared server and due to this reason CUDA is installed in a custom directory.