I’m currently unable to utilize my GPU as I get this error any time I try to use CUDA functions:
UserWarning: CUDA initialization: CUDA unknown error - this may be due to an incorrectly set up environment, e.g. changing env variable CUDA_VISIBLE_DEVICES after program start. Setting the available devices to be zero. (Triggered internally at  /pytorch/c10/cuda/CUDAFunctions.cpp:109.)
Funny thing is that I don’t recall having upgraded my drivers or changing anything related to pytorch in the last few months, it used to be working fine.
I’ve tried re-installing pytroch multiple times (through conda), testing out different cuda toolkit versions but it doesn’t seem to help. I’ve also tried re-booting.
Here’s the collect_env output:
Collecting environment information...
/home/nick/miniconda3/lib/python3.8/site-packages/torch/cuda/__init__.py:52: UserWarning: CUDA initialization: CUDA unknown error - this may be due to an incorrectly set up environment, e.g. changing env variable CUDA_VISIBLE_DEVICES after program start. Setting the available devices to be zero. (Triggered internally at  /pytorch/c10/cuda/CUDAFunctions.cpp:109.)
  return torch._C._cuda_getDeviceCount() > 0
PyTorch version: 1.8.0
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.2 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Clang version: Could not collect
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.8.10 | packaged by conda-forge | (default, May 11 2021, 07:01:05)  [GCC 9.3.0] (64-bit runtime)
Python platform: Linux-5.11.0-25-generic-x86_64-with-glibc2.10
Is CUDA available: False
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: GeForce GTX 1070 Ti
Nvidia driver version: 450.119.03
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.21.0
[pip3] numpy-quaternion==2021.3.17.16.51.43
[pip3] pytorch-lightning==1.3.1
[pip3] pytorch-lightning-bolts==0.3.2
[pip3] torch==1.8.0
[pip3] torchmetrics==0.3.2
[pip3] torchvision==0.9.0
[conda] cudatoolkit               11.0.221             h6bb024c_0    nvidia
[conda] numpy                     1.20.1                   pypi_0    pypi
[conda] numpy-quaternion          2021.3.17.16.51.43          pypi_0    pypi
[conda] pytorch-lightning         1.3.1                    pypi_0    pypi
[conda] pytorch-lightning-bolts   0.3.2                    pypi_0    pypi
[conda] torch                     1.8.0                    pypi_0    pypi
[conda] torchmetrics              0.3.2                    pypi_0    pypi
[conda] torchvision               0.9.0                    pypi_0    pypi

Also, nvidia-smi and nvtop seem to be working fine, here’s the output:
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.119.03   Driver Version: 450.119.03   CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce GTX 107...  Off  | 00000000:08:00.0  On |                  N/A |
|  5%   49C    P0    38W / 180W |    524MiB /  8113MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1135      G   /usr/lib/xorg/Xorg                 83MiB |
|    0   N/A  N/A      1832      G   /usr/lib/xorg/Xorg                212MiB |
|    0   N/A  N/A      1970      G   /usr/bin/gnome-shell              170MiB |
|    0   N/A  N/A      2156      G   ...wnloads/Telegram/Telegram        5MiB |
|    0   N/A  N/A      2458      G   .../debug.log --shared-files       32MiB |
+-----------------------------------------------------------------------------+


Thanks in advance for any help.