NightRain:

Is there a more efficient way to do this, considering multiple processes applying a random transformation every epoch to the objects where their order changes due to the shuffling?


I think one good approach would be to use the worker_id and seed all 3rd party libraries with this id.
You can get the id via torch.utils.data.get_worker_info() 5.
PyTorch itself should yield deterministic data samples, if you’ve properly set the seed before.
However, if you are using e.g. the Python random library or numpy in the data loading pipeline, you would have to seed them as well using the aforementioned approach.