Since you are using PIL to load and process the images, you could try to install PIL-SIMD, which is a replacement for the standard PIL lib and might give you a faster preprocessing pipeline.
The np.concatenate operation doesn’t look alright, as the transforms should return a tensor already.
Could you swap it for torch.cat?
Also, the number of workers is currently quite high for train_loader. I would recommend to play around with this number (lower it a bit) and check, if you can get rid of the bottleneck.
That being said, did you make sure that the data loading is actually the bottleneck?
Have a look at the ImageNet example 12 to see, how to time the data loading.