It’s a very large network, so it is going to be very dificult for me to reproduce the issue with a short piece of code.
But I’m getting closser to the issue, as changing the tensor type to double instead of float using:
torch.set_default_tensor_type('torch.DoubleTensor')
solves the issue and allows me to get deterministic results.
I’m still searching why is that happening.