Batches are mainly a way to accelerate training, think about using batch_size=1.
You start from a blank state, accumulate a variable number of events into a fixed size state, and then use it for classification or whatever.
If that’s unclear, 3d tensors with additional time dimension are used to feed events in, instead of “batches of events”.
Your “state preservation” approach, even if it works, won’t allow making out-of-sample predictions.



 rohit901:

So take-away here is that seq length doesn’t actually matter and basically since we can’t sometimes take huge sequence at once in memory we have this seq length parameter?


I guess you refer to “seq length” in some data loader implementation, then this is mostly correct. You also don’t treat your data as one “huge sequence” because a model trained this way won’t generalize to other sequences.