Hello, I just came across this topic because I’m actually trying to do batch normalization for multivariate time series data, and I did it on the features following exactly the same method you described :
bn = nn.BatchNorm1d(1024)
x = torch.randn(64, 256, 1024)
x = x.permute(0, 2, 1)
output = bn(x)
However I applied this in the forward method of the LSTM class I created, and therefore I give it as an input to the LSTM, the results are just perfect for me, but I still have one question :

Since Batch normalization applies to each layer in the LSTM I have the feeling it is not the case following what I just did, because I just add a few line in the forward method of the LSTM, and I don’t know it really applies to each layer or just the input, because it looks like I only apply it to the input.
Here’s the part I added in the forward method :
Capture d’écran 2020-05-11 à 16.17.30950×350 23.3 KB