Well even if that’s the case, those operations takes usually lot of time. I was using a dataset in which i choose random samples to create grountruth in a unsupervised way. Forward-backward takes 3 s to be done. This preprocess time of picking 2 samples and making gt was taking other 3 sec. In the end preprocess that way was very time consuming for doing it online and i had to fix it beforehand.
What i mean is that all those operations are very time consuming and they slow down the training process. That’s the main reason why it’s not implemented inside the dataset.
In addition, you are thinking in very simple cases, like feeding a net with images which just need few operations like cropping or rearranging, but there are much more complex nets in which you can’t generalize those transformations or which are computationally very expensive