Since you have multiple GPUs, you could use nn.DataParallel to utilize all or some of them.
Have a look at this tutorial 11 to apply it.
Basically your batch will be split into chunks in the batch dimension and pushed to all specified devices.
Also to speed up the data loading you should use multiprocessing in your DataLoader by setting num_workers>0.