My biggest concern was the lack of reproducibility that could arise from each image being randomly cropped. I will now be centre cropping my images instead to work around this.
You are right, my workload is quite large and comparatively the loading is not that bad (maybe 5% of total running time). Thanks for your advice ptrblck !