Your IO might be the bottleneck.
Are you storing the data on a local SSD or on an HDD / network drive?
You could try to time the data loading using the ImageNet example 2.
As a small side note: Variables are deprecated since PyTorch 0.4.0, so if you are using a newer version (which I would highly recommend  ), you can just use torch.tensor instead. Also, you don’t need to convert your data using type(dtype) and then .cuda(). In the latest version, just push your data onto the GPU using data = data.to('cuda:0') (or any other GPUid).