Hi albanD,
I see, I think it’s different is because I didn’t set torch.backends.cudnn.deterministic = True even though I have set the torch.manual_seed. I’ll try this and remove some operations in the forward to get a comparable comparison. Thank you very much for your reply!