How does nn.BatchNorm1d() work in eval() mode? I looked up the source code, but this points to F.batch_norm, but this points to torch.batch_norm (which has no file I can look up).
Fundamentally, I want do something very simple.
Suppose, I have input x, which is passed into nn.BatchNorm1d(x) to get y1 during eval() mode. I want to recreate y by using the parameters (gamma, bias, running_mean, running_var - where gamma and bias are stored inside the module’s parameters variable) that are inside nn.BatchNorm1d via the following:
y2 = gamma * (x - running_mean) / (sqrt(running_var)) + bias
However, y1 != y2. Am I missing something?
Thank you!