This always works for me (Win10, Cuda 10.1, Python 3.7.2, PyTorch 1.0.1, NVIDIA GTX 1050 Ti):
import torch
torch.cuda.current_device()

but this always fails for me:
import torch
torch.cuda.is_available()
torch.cuda.current_device()  # fails here

@Mohamed_Ghadban, how can I access the NVIDIA Nsight Options? Thanks in advance