I don’t think batchnorm should be used with variable-length time series data(u can google for why, and I’ve had bad experience in cases very similar to yours). Using layernorm for LSTM should improve the performance in most cases(sometimes not very significant, though), but u must use it correctly, see https://github.com/pytorch/pytorch/blob/master/benchmarks/fastrnns/custom_lstms.py 200.