nnss:

In my code the seed is always set to 1 for both models so that I can reproduce the error.


Could you rerun the codes with different seeds?
We would need a code snippet to reproduce this issue, i.e. the model definition as well as the optimizer, input shapes etc. so that we could run the code with random tensors.