If you sort your sequences so that you’ll have adjacent frames, what should a batch contain?
E.g. if you specify batch_size=3, you would get a batch containing frames [1_1], [1_2], [3_4].
Would this be OK?
What if you’ll get [1_1], [1_2], [1_2]? Would this still be alright?
Or should the batch only contain sequential frames of one sequence, e.g. [1_3], [1_4], [1_5]?