Solved by ruotianluo in post #5 
              
              
                The reason why dataloader cannot fetch some batch in the middle is once the dataloader is created, it’s prefetching the data according to the internal order inside the dataloader. 
So if you want to get i-th batch or get a random batch, you have to tell the loader what you want(so it’s not actually …