The error
CUDA initialization: CUDA unknown error 

is unfortunately not very helpful.
Could you check dmesg for any XID error codes and post them here?
Also, could you check, if docker containers with CUDA11 and PyTorch work fine on your machine?