Hey @aatrey,
Thanks for the feedback!

I was wondering if there is a way to remove the last sequences once we hit the  idx + self.seq_length  condition instead of appending 0s as it does now.

I think there are several ways for solving this. First two that come to my mind:

use Torch pad_packed_sequence 9 for generating batches of sequences of the same length starting from sequences of variable length, and Torch pack_padded_sequence 4 for recovering the sequences in their original form (you can think of these operations as the inverse of each other);
since the root problem is that standard DataLoaders do not handle batches of variable size, write your own DataLoader with a custom collate_fn that handles variable size inputs (take a look here 4).

Hope these suggestions can be a good start for your troubleshooting, sorry if my answer couldn’t be more detailed but it has been quite a while since the last time I’ve worked with LSTM in PyTorch, and I don’t want to give you any bad advice.
Cheers