And the culprit is …
out = F.relu(self.bn1(self.conv1(x)))    <- track_running_stats = TRUE
out = self.layer1(out)                   <- track_running_stats = TRUE
out = self.layer2(out)                   <- track_running_stats = TRUE
out = self.layer3(out)                   <- track_running_stats = TRUE
out = F.avg_pool2d(out, out.size()[3])
out = out.view(out.size(0), -1
 out = self.linear(out)


So, the tracking of stat of the last layer of the resnet (sorry iy is written BasicBlock but the layers are defined as:
    self.layer1 = self._make_layer(block, 16, num_blocks[0], stride=1, track_run_stat=True)
    self.layer2 = self._make_layer(block, 32, num_blocks[1], stride=2, track_run_stat=True)
    self.layer3 = self._make_layer(block, 64, num_blocks[2], stride=2, track_run_stat=True)

and the first BN

self.bn1 = nn.BatchNorm2d(16,track_running_stats=True)

So far so good, what I should conclude Dr. Watson?