Hi, thank you for your reply.
Do you have any suggestion on how it should be ordered? I think only the kernel computation order matters. I tried to reorder the kernel computation (and the bias assignment up front instead of at the end), but with little success.
edit: well of course, the back-end of PyTorch probably uses GEMM (or something similar) for convolution, which in case of GEMM might even modify the number of multiplications or accumulations with optimized MatMul routines.
In the end I do matter about these few rounding errors because, I tried to substitute some PyTorch Conv2d layers with my own reference for some experiments, but the network outcome was affected (accuracy and loss).
I guess there is no easy solution for this problem