Solved by Redcxx in post #2 
              
              
                Ah I found it, the batch norm layer’s input dimension should be 1 instead of n_neurons, it is the number of channels rather than the number of features.