Solved by tom in post #2 
              
              
                Changing the batch size may change how computation is organized and deviations within numerical accuracy are expected. 
For things like convolutions you might even have specialized “inference” kernels for batch size one, but you can even see this with much simpler operations: 


If you want CPU comp…