Whether or not this is a problem depends a lot:

Convolutions and linear layers and other reductions “average” errors in the input. If these are independent, then the errors won’t accumulate.
For other operations, this is much more tricky, e.g. taking exponentials in softmax to get probabilities and then log (e.g. for KL divergence / “cross entropy” etc.) can run into precision problems, in particular with very small probabilities.
Finally, while the ops in a single model run often are not problematic, this becomes different when you train models in millions or billions of iterations: in the course of a training run, these errors can accumulate quite dramatically, so the differences may mean that you cannot expect to reproduce the same weights as the end results when running the exact same code. This is quite annoying for reproducing scientific results etc.