If your data augmentation relies heavily on PIL, you could try to install PIL-SIMD 9 as a drop-in replacement, which should speed up your code.
Also, could you profile your code to see, where the bottleneck if your data loading is?
If you are IO bound, you could move the data onto a SSD if that’s not already the case.