Solved by KFrank in post #2 
              
              
                Hi Tobias! 

As you suspect, this is very likely floating-point round-off error. 
As the size of a tensor changes, pytorch can change the details of 
how it pipelines tensor operations, both on the cpu and gpu.  This 
can cause operations to be reordered in a way that is mathematically 
equivalent, …