I don’t know how you are reshaping the input or activation before feeding it into the nn.LSTM, but note that the sequence length is defined in the temporal dimension not the batch dimension.
I.e. by default the nn.LSTM module expects an input in the shape [seq_len, batch_size, nb_features], while it seems you are trying to increase the batch size and are running out of memory.