Hey!
If I understand it correctly, when training RNNs using mini batch sgd, the elements in one batch should not be sequential. Rather, every index throughout the batches corresponds to one  sequence. I can see that this makes sense when one has multiple sequences to train on.
Currently I’m working on a problem where I have only 1 ongoing time series, no individual sequences. Is it common practice to artificially create sequences by splitting the training data? Or should I train with batch size 1 and consider my whole training set as one sequence?
It would be cool if someone could confirm my understanding of the matter and/or maybe give some some tips on best practices for this sort of situation.
Thanks!