@ptrblck as I analyzed further even without learning parameters the output from a batch norm will have standard deviation of 1 and mean of 0 which rises the question of why do we need learnable parameters at the first place?
import torch
import torch.nn as nn

# Without Learnable Parameters
m = nn.BatchNorm1d(10, affine=False)
input =  1000* torch.randn(3, 10)
print(input)
output = m(input)
print(output)
print(output.mean()) # will be close to 0
print(output.std()) # will be close to 1