For 2D Batches it works great. However, when we use fully connected layers for 3D inputs of shape (N=batch size,L=sequence length,C=input size) we have to transpose 2 times to use BatchNorm1D after each linear transformation. Because each linear layer acts upon dim=-1 (our features) and BatchNorm can act only on dim=1, 2 transpositions must be done: before and after the BatchNorm. For example, a fully connected network of 3 layers is given below with BatchNorm before each non-linearity.
class FC_layer(nn.Module):
    def __init__(self,input_size_FC1,output_size_FC1,output_size_FC2,output_size_FC3):
        super(FC_layer, self).__init__()
        
        self.linear_layer1 = nn.Linear(input_size_FC1,output_size_FC1)
        self.normalization1 = nn.BatchNorm1d(output_size_FC1)
        self.linear_layer2 = nn.Linear(output_size_FC1,output_size_FC2)
        self.normalization2 = nn.BatchNorm1d(output_size_FC2)
        self.linear_layer3 = nn.Linear(output_size_FC2,output_size_FC3)
        self.normalization3 = nn.BatchNorm1d(output_size_FC3)
                        
    def forward(self,x):
    """ 
        x : tensor of shape (N_batch, N_sequence, N_features)
    """
        
        f = nn.ReLU() # Activation functions
        g = nn.LogSoftmax(dim=2)
               
        x_lin1 = self.linear_layer1(x) # Apply linear transformation
        x_lin1 = torch.transpose(x_lin1,1,2) # Transpose for BatchNorm1d
        x_lin1_norm = self.normalization1(x_lin1) # Normalize
        
        layer1_out = f(x_lin1_norm) # Apply non-linearity
        
        layer2_in = torch.transpose(layer1_out,1,2)  # Transpose for next linear transformation
        
        x_lin2 = self.linear_layer2(layer2_in)
        x_lin2 = torch.transpose(x_lin2,1,2)
        x_lin2_norm = self.normalization2(x_lin2)  
        
        layer2_out = f(x_lin2_norm) 
        
        layer3_in = torch.transpose(layer2_out,1,2) # Transpose for next linear transformation
        
        x_lin3 = self.linear_layer3(layer3_in)
        x_lin3 = torch.transpose(x_lin3,1,2)
        x_lin3_norm = self.normalization3(x_lin3)  
        
        layer3_out = g(x_lin3_norm)
        
        output = torch.transpose(layer3_out,1,2)      
                
        return output

Instead, if BatchNorm1d could act upon the final dimension, we wouldn’t need to worry about the dimensions conversions.
I couldn’t find a way to do this in the sequential container, but you are suggesting that I could write a permutation module with no parametes and use this in the sequential container. Right? I am worried a about  the necessity to do a copy()  or replace the tensor with its transposed version. Also about the contiguous requirment of other pytorch functionalities.