You could also try something new:
Use some pytorch layers for the data augmentation. But set them to requires_grad=False. Then each n steps you could change the properties of the augmentation randomly.
This would support DataParallel and thus multiple GPUs.