If you have to make sure the same randomly augmented images are used and each fold contains exactly the same samples, I would recommend to create this particular dataset in a separate script and store each fold as PyTorch tensors using torch.save.
In your training script you could simply load the folds and train your model for the specified number of epochs.
Alternatively, it should be possible to properly seed the code and create the same folds in your training script, but you would have to be very careful and would have to verify each step to avoid accidentally calling into the random number generator (which would thus change e.g. the augmentation).