Solved by nbortych in post #3 
              
              
                I’ve solved the problem by hacking together the Distributed sampler and the batch_sampler function from the migration tutorial. 
Note that this implementation is for a seq2seq problem and the dataset is a tuple with 4 values: source seq, length of source seq, target seq and length of the target seq.…