Solved by divinho in post #2 
              
              
                Okay I had to set drop_last=True. 
Also I was making the mistake of assuming __init__ would be called again when the dataset is recreated for each dataloader process but that is not the case.