Thank you for the comments, ptrblck!
I think I found the reason for the problems above.
Originally I was using BatchNorm1ds like in the first code bellow.
With the first code, I had to use track_running_stats = False in order to have the reasonable outputs.
Unless, the outputs were always 0.
The shape of the initial input is (N,400,3): Length 400, channel 3
After nn.Linear layer, it becomes (N,400,32)
Then I wanted to pass it through nn.BatchNorm1d.
I initially thought that the batch norm should be done on 3rd dimension, but as BatchNorm1d takes inputs of shape (N,C,L) I transposed 2nd and 3rd dimension like the code bellow.
# First code. I had to use track_running_stats = False in order to avoid outputs with all-zero value.
class someModel(torch.nn.Module):
    def __init__(self):
        super(someModel, self).__init__()
        self.input_size = 3 
        self.dims = 32
        self.relu = torch.nn.ReLU()
        self.fc1 = torch.nn.Linear(self.input_size, self.dims)
        self.bn1 = torch.nn.BatchNorm1d(self.dims)

    def forward(self, x):
        # x shape: N,400,3 
        x = self.relu(self.bn1(self.fc1(x).transpose(1,2))).transpose(1,2)
        return x # N,400,32

After reading your comment, I removed transpose operation and batch norm on 2nd dimension (400) rather than 3rd dimension(32) like code bellow
# Second code. I do not have to set track_running_stats = False, anymore.
class someModel_2(torch.nn.Module):
    def __init__(self):
        super(someModel_2, self).__init__()
        self.input_size = 3 
        self.dims = 400 # CHANGED HERE 32 -> 400
        self.relu = torch.nn.ReLU()
        self.fc1 = torch.nn.Linear(self.input_size, self.dims)
        self.bn1 = torch.nn.BatchNorm1d(self.dims)

    def forward(self, x):
        # x shape: N,400,3 
        x = self.relu(self.bn1(self.fc1(x))) # removed transpose
        return x # N,400,32

The problems are now gone and looks second way is right.
And also, don’t need to set track_running_stats = False.
I have one last question though.
I am not familiar with these 3D Tensors (N,C,L) or (N,L,C)…
Why Batch norm should be done on the 2nd dimension of size 400 (L) like in the second way?
Should it not be done on 3rd dimension of size 32 (C ) as it become (N,400,32) after the nn.Linear layer?
Can you or someone explain in easy way?
Thank you once again for fixing my problem, ptrblck!