How about this?
Just use BatchFirst is False, but got the batch data as B x S. Then, you just need to transpose once in your RNN cell. Then, that’s it.
def forward(self, input, seq_lengths):
    # Note: we run this all at once (over the whole input sequence)
    # input shape: B x S (input size)
    # transpose to make S(sequence) x B (batch)
    input = input.t()
    batch_size = input.size(1)

Then, everything is OK. For the entire code, please check out at https://github.com/hunkim/PyTorchZeroToAll/blob/master/12_4_name_classify.py 79.