Hm, do you maybe have a very slow disk, e.g., a regular spinning hard drive?
Maybe try one of my toy example notebooks. I just ran those and get ~82% utilization with ResNet-50 on MNIST and 99% with VGG-16 on Cifar


https://github.com/rasbt/deep-learning-book/blob/master/code/model_zoo/pytorch_ipynb/convnet-resnet50-mnist.ipynb
https://github.com/rasbt/deep-learning-book/blob/master/code/model_zoo/pytorch_ipynb/convnet-vgg16.ipynb 1

(you wouldn’t need anything extra to run them).