I tried setting up my data as (sequences, timesteps, features) and then selecting timesteps 1…100 for the first batch, then 101…200 for the second batch, but pytorch complained that the data wasn’t contiguous.
So I changed my dataloader to produce data as (timesteps, sequences, features) and the problem went away.
If timesteps is the first dimension then you can run sequential batches efficiently without having to copy the data.