Thanks a lot for your response!
Actually the reason I asked this question (which I should included above) is because I was not observing what you mentioned. So let me put forth my example used:
I have a simple model that looks like this:
class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3,2,kernel_size=3, bias=False)
        self.bn = nn.BatchNorm2d(2, track_running_stats=True)
        self.fc1 = nn.Linear(18, 10, bias=False)

    def forward(self, x):
        x1 = self.conv1(x)
        print("conv output")
        print(x1)
        # amean = np.mean(x1.data.numpy(), axis=(0,2,3))
        # print(amean)
        # astd = np.std(x1.data.numpy(), axis=(0,2,3))
        # print(astd)
        # x11 = x1.data.numpy()
        # for i in range(2):
        #     x11[:,i,:,:] = (x1.data.numpy()[:,i,:,:] - amean[i]) / astd[i]
        # print(x11)
        x2 = self.bn(x1)
        print("bn output")
        print(x2)
        x3 = F.relu(x2)
        print("relu1 output")
        print(x3)
        x4 = x3.view(x3.size(0), -1)
        x5 = self.fc1(x4)
        print("fc output")
        print(x5)
        x6 = F.relu(x5)
        print("relu2 output")
        print(x6)
        return x6

    def num_flat_features(self, x):
        size = x.size()[1:]  # all dimensions except the batch dimension
        num_features = 1
        for s in size:
            num_features *= s
        return num_features

torch.manual_seed(0)
net = Net()
net.bn.bias.requires_grad=False
net.bn.eval()


Now I pass in one training sample from the forward function if I check the output of bn, this is what it prints:
bn output
tensor([[[[-1.6799, -0.0496,  0.2127],
          [ 0.3922,  1.3428,  0.0598],
          [ 0.3674,  0.3883, -1.0339]],

         [[ 0.4344,  0.1697, -0.1216],
          [ 0.1510, -0.0127, -0.1253],
          [-0.0596, -0.1495, -0.2863]]]], grad_fn=<NativeBatchNormBackward>)


and output of the previous layer (conv) is:
 tensor([[[[-0.0403,  0.0103,  0.0185],
          [ 0.0240,  0.0535,  0.0137],
          [ 0.0233,  0.0239, -0.0202]],

         [[-0.1044, -0.1664, -0.2347],
          [-0.1708, -0.2092, -0.2356],
          [-0.2202, -0.2412, -0.2733]]]], grad_fn=<MkldnnConvolutionBackward>)


if eval did not calculate anything from the current batch, then the output of bn should not have been -1.6799 at the first index, for example. The -1.6799 is obtained when normalizing the output of conv. In the forward function the commented out code shows the formula used to obtain the same result. If I check the first index of x11 it also is -1.6799.
Could you please explain why I see this? If I replace eval() with train() it is the same result. This is the one and only training sample used (one batch) and if it did not use current batch stats, it should have some other different output I presumed because no previous mean and variance was calculated as this is the first time the model is provided with an input. But it looks like even using eval(), it is using the current batch to make the output.
Thanks!