Out of curiosity, are you doing any augmentation on 720px images?  From what I understand data augmentation is not multi-threaded, even though loader is multi-threaded.
Source: https://sagivtech.com/2017/09/19/optimizing-pytorch-training-code/ 1