Hi,
In general, setting the seed will give you consistent random number for a given version of pytorch.
Having reproducible results on different hardware is (almost) impossible as different hardware will handle floating point ops differently.
The problem with training with nn training is that, one layer gives an error which is at the level of floating points errors. The next layers will amplify this error until you compute the loss that will be slightly different (depends on the depth). The backward pass will amplify these errors again and the computed gradients will be slightly different. Finally the new weights after the gradient update will be significantly different. If you repeat this 10 times with 10 batches of data, you get results that are significantly different.
Such differences are expected and well designed neural networks have a stable enough training that such problems do not matter.