Thank you ptrblck, that did the trick :).
Although I’m wondering now how PyTorch implements batch norm. From what I have learnt two parameters are trainable in batch norm: scale and bias applied to the normalized vector. But also batch’s mean and standard deviation are sometimes trainable. If they indeed are trainable then I don’t think it made any difference whether we run a single or many instances through the network. So the question is: does PyTroch, when using batch norm, also trains mean and standard deviation?