Check this 4
and Pytorch note 1 on the subject.
There are more modules that you need to fix their seed.
You have to fix the seed of any module that may introduce randomness.
I recommend to start with num_workers=0 or 1 to use the main process (for the dataloader). Once you obtain reproducible results, you can move to > 1.