I don’t think you are using GPU. If GPU is used, you’ll see something like this:

gpu2.png732×438 52.8 KB

To check whether you have CUDA access inside your code, you can use:
>>> print(torch.cuda.is_available())
True

If it return false, I highly recommend to check the CUDA version you have installed and make sure that you have installed corresponding PyTorch version.
Check CUDA version:
>>> nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2015 NVIDIA Corporation
Built on Tue_Aug_11_14:27:32_CDT_2015
Cuda compilation tools, release 7.5, V7.5.17

Install Pytorch 1.0 for CUDA 8.0 and above, follow this link 4
$ conda install pytorch torchvision cuda80 -c pytorch
Install previson verison of Pytorch for CUDA 9.0 and below, follow this link 2
$ conda install pytorch=0.4.1 cuda80 -c pytorch