Ho on CPU.
I guess you double checked that you set all the random seeds: python random, pytorch, numpy, other stuff if you use more.
Do not use multiprocessing.
Otherwise the rest should be quite deterministic on CPU.