There are a couple of things to unpack:


Sequences of 10,000 are extremely long. Sure, LSTMs proud themselves to have long-term memory but 10k time steps seems awfully long. I’m not saying that it won’t work, it’s just something too keep in mind.


Since the range of you sequences varies a lot, have a look at BucketIterator 4 to generate batches where all sequences within one batch have the same or at least very similar size.


Since batches can still have sequences of different lengths, you should have a look at pack_padded_sequences 5.


Once the batches are right setting up the network model is relatively straightforward.