I want to fix the randomness of the training for compare the performance of different super-parameters. Now I fix the random seed of random.seed(), numpy.random.seed(), torch.manual_seed() and torch.cuda.manual_seed(), but I found I still can’t get the same result in different training process, even I set the p of dropout as 0. Now the result seems to be similar but there is still some small difference, especially the AUC value.
I guess the SGD may still start from different random state but I can’t check well. And I still want to use dropout with non-zero p in the training but I don’t know how to keep it in the same random state.  Please give me some advice if you have idea about this question. Thank you !