I am providing this for my better understanding. So far I understood number of batches tracked will increase with every batch.
 if self.training and self.track_running_stats:
            # TODO: if statement only here to tell the jit to skip emitting this when it is None
            if self.num_batches_tracked is not None:
                self.num_batches_tracked += 1
                if self.momentum is None:  # use cumulative moving average
                    exponential_average_factor = 1.0 / float(self.num_batches_tracked)
                else:  # use exponential moving average
                    exponential_average_factor = self.momentum

weight and bias are learnable parameter of choice, and they can scale the output activation range effectively else it will be std (-1,1) as we saw with the mean of 0.
What are the running mean and running var?
They are used for the moving average calculation
Two types of moving averages we support as you can see:

exponential moving average
cumulative moving average

At the very end they will smoothen the curve which is what the batch norm do.

So the running mean and running var are calculated as average, not learned like Parameter or some parameter in the nn.Linear module using the loss function.