The printed differences seem to be in the range 1e-6, which is most likely due to the limited floating point precision.
Could you explain the issue regarding the changed predictions a bit more?
I.e. how large is the absolute difference after which layer and how many training iterations?
Since these small errors might add up, I wouldn’t expect to get exactly the same outputs after some epochs of training.
However, neither would I expect that one of the machines always produces as “good” model, while the other one fails.