Maybe a useful feature when inferring a model would be batch size so that when forwarding an input of a given minibatch size, batchnorm layers know to carry on averaging until the total batch size is reached. I think tensorflow for swift does this