The internal .training attribute determines the behavior of some layers, e.g. batch norm layers.
If you call model.train() or model.eval() (or model.bn_layer.train()), this internal flag will be switched.


If you are using a single sample as your batch, you might consider using other normalization lazers, e.g. InstanceNorm.