Thanks for the reply! I have seen the documentation on reproducibility.
In this case, to begin with, I am only looking for reproducibility for my particular set-up (hardware and software). I am indeed using a shell script to launch the jobs, but I thought it would be neater if I could do this all within one script with PyTorch responsible for the distribution.