blackberry:

I should get the same output regardless of the batch size.
Is it possible with a network with batchnorm?


This won’t be possible with track_running_stats=False, since the batchnorm layers will calculate the current batch statistics, which will vary based on the batch size.
However, track_running_stats=True should give you the same outputs (up to floating point precision, if non-deterministic methods were used or if you haven’t used the setup from the reproducibility docs 5).
How large is the absolute difference with track_running_stats=True?