Schema for feeding RNN training data
<SCODE>             t=0  t=1  t=2  t=3  t=4  t=5  ...  t=598  t=599
sample       |---------------------|
sample            |---------------------|
sample                 |-----------------
...
sample                                          ----|
sample                                           ----------|
<ECODE> <SCODE>             t=0  t=1  t=2  t=3  t=4  t=5  t=6  t=7  ...  t=598  t=599
sample       |-----------|
sample                      |-----------|
sample                                     |-------
...
sample                                                    -----------|
<ECODE> Thanks
d10genes
jekbradbury
Thanks for the answer. I’m using time series data rather than LM but it sounds like the 2nd schema is still preferred. Is it important that the sequences are trained in order so that the relevant hidden state is reused, or is it common to shuffle them? It seems the hidden state may not be used anyways if they’re fed in parallel batches.
d10genes
For truncated BPTT training, it’s important that batches be processed in order so that hidden states are preserved.
jekbradbury
ecolss
jekbradbury