Digbalay_Bose:

So the base_seed generated by the main process is different than the seed fixed manually using torch.manual_seed?


Yes, the base_seed is generated from drawing a random number, so it is dependent on / defined by the RNG state (and thus the manual_seed) but not identical to it.



 Digbalay_Bose:

Also, if I interpret it, for each epoch, the workers will be assigned a new seed base_seed + worker_id. And the data augmentations will be based on the new worker seed?


One thing to keep in mind is that if enabled, the shuffeling of the dataset (i.e. which things to stick in a minibatch) is the done in the parent.
The randomness in the dataset (and potentially the collation function) like augmentations (in the dataset) are indeed from in the worker based on the new worker seed. All this assuming that you use PyTorch’s random functions (which you should and you need to be super-careful to properly initialize your RNG if you don’t).
All this said, there is a good argument to be made to do the random augmentation after batching on the GPU if possible. Solves not only any confusion around randomness but is likely much more efficient, too.
Best regards
Thomas