Thanks for the useful post, but the actual problem was my code.
I was applying huge augmentations before resizing the image, not vice versa, - resolution of image on which augmentations was performed was like 800x600 instead of 224x224, and that caused the bottleneck.
Changing the order of augmentations and resizing fixed my problem, for sure