I went through the code of batchnorm.py and found this statement in batchnorm.py:
line 78 in pytorch 1.3.1:
        return F.batch_norm(
            input, self.running_mean, self.running_var, self.weight, self.bias,
            self.training or not self.track_running_stats,
            exponential_average_factor, self.eps)

Maybe this statement should be  self.training and self.track_running_stats  according to the documentation  If track_running_stats is set to False, this layer then does not keep running estimates, and batch statistics are instead used during evaluation time as well.
F.batch_norm should only receive training=True if self.training=True and self.track_running_stats=True