In fact I have fixed the random seed using the code:
torch.manual_seed(0)
torch.cuda.manual_seed_all(0)
random.seed(0)
np.random.seed(0)
torch.backends.cudnn.deterministic = True

And I thought that with fixed random seed, the 2 approaches should produce the same result. But turns out that it’s not the case.
But what should the randomness come from if I have already fixed the seed?