If you use multiprocessing (used by default if you run on cpu) or multigpu or some non-deterministic gpu operations, the accumulation between different workers can happen in a different order. Remember that for floats, (a+b)+c != a+(b+c) and so this kind of small errors can appear.