Are you able to create a dummy tensor on the GPU via:
x = torch.randn(1, device='cuda')

Also, was this GPU working before? If so, did you update any drivers or other libs without restarting the machine?