stases:

What bothers me is how in general data augmentation works, meaning will I augment my data, save it to HDD and then load it, or is it done “per batch”, stored temporarily?


Data augmentation is applied on the fly for each batch during training.
If you are using a Dataset and pass it to a DataLoader with multiple workers, the data loading and processing would be executed in the background while the model is training.
I would have a look at torchaudio 11 which ships with some transformation.