I have a similar problem.
root@4a2115485a52:/host/mlbench/pytorch# python3
Python 3.6.9 (default, Jan 26 2021, 15:33:00)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.is_available()
False
>>> quit()
root@4a2115485a52:/host/mlbench/pytorch# nvidia-smi
Mon Aug 30 02:14:13 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.119.03   Driver Version: 450.119.03   CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro P2000        Off  | 00000000:01:00.0  On |                  N/A |
| 48%   37C    P8     6W /  75W |    119MiB /  5055MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+


But I am in a docker container. modprobe is not installed in the container.
Running modprobe on the host gives an error:
peter@mouse:~/ml/mlbench$ sudo rmmod nvidia_uvm
rmmod: ERROR: Module nvidia_uvm is not currently loaded
peter@mouse:~/ml/mlbench$ sudo modprobe nvidia_uvm
modprobe: FATAL: Module nvidia_uvm not found in directory /lib/modules/5.11.0-27-generic

NVdrivers, CUDA, cuDNN versions inside container:
cuda path:/usr/local/cuda
NVDRV:450.119.03,CUDA:10.2,cuDNN:7.6.5.32-1

CUDA is not installed on the host.
Restarting the host machine (with Ubuntu 20.04) does not solve the problem.