Solved by fs4ss1 in post #2 
              
              
                I think that you can do the follow in the training loop: 
def forward(self, x):
    x1 = self.BatchNorm(x[:-1])
    x = torch.cat((x1, x[-1]), dim = 1)
    .
    .
    .
    return x